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United  States 

General  Accounting  Office 

Washington,  D.C.  20548 


Program  Evaluation  and 
Methodology  Division 

B-256574 

November  25, 1994 

The  Honorable  Marilyn  Lloyd 
Chairman,  Subcommittee  on  Energy 


The  Honorable  Tim  Valentine 
Chairman,  Subcommittee  on  Technology, 

Environment,  and  Aviation 

The  Honorable  Tom  Lewis 
Ranking  Minority  Member,  Subcommittee 
on  Technology,  Environment,  and  Aviation 
Committee  on  Science,  Space,  and  Technology 
House  of  Representatives 

You  asked  us  to  consider  how  the  national  laboratories  of  the  U.S. 
Department  of  Energy  (doe)  can  best  be  focused  to  help  solve  the 
problems  our  nation  faces  during  the  current  decade.  As  a  beginning,  we 
have  developed  an  inventory  of  the  human  and  capital  resources  housed  in 
the  national  laboratories  that  will  provide  baseline  data  for  future  reports 
on  a  number  of  doe  laboratory  policy  issues.  This  report  addresses  the 
Committee’s  interest  in  the  current  balance  of  the  research  effort  in  the  10 
laboratories’  research  programs.  It  examines  the  extent  to  which  the 
national  laboratories  are  engaged  now  in  basic  and  applied  research  or  in 
research  related  to  commercial  product  development. 


Background 


The  Department  of  Energy’s  multiprogram  laboratories  have  had  missions 
that  are  national  in  scope  since  their  inception  during  World  War  II.  The 
original  laboratories — Lawrence  Berkeley  (Calif.),  Los  Alamos  (N.  Mex.), 
and  Oak  Ridge  (Term.)— were  established  as  government-owned, 
contractor-operated  institutions  to  apply  the  productive  capability  of 
private  industry  to  the  development  of  atomic  weapons.1  The 
weapons-development  mission  continued  during  the  cold  war,  and  six 
additional  laboratories— Argonne  (Ill.),  Brookhaven  (N.Y.),  Sandia  (N. 
Mex.  and  Calif.),  Idaho  Engineering  (Idaho),  Lawrence  Livermore  (Calif.) 
and  Pacific  Northwest  (Wash.)— were  created  between  1946  and  1965  to 


1 9950224  040 


‘Ernest  Orlando  Lawrence  founded  the  Radiation  Laboratory  for  basic  scientific  research  on  the 
University  of  California  at  Berkeley  campus  in  1931.  It  was  funded  under  government  contract  in  1942. 
The  laboratory  was  renamed  Lawrence  Radiation  Laboratory  after  his  death  in  1958  and  later  called 
Lawrence  Berkeley. 
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foster  civilian  applications  of  nuclear  technology.2  A  10th  laboratory,  the 
Solar  Energy  Research  Institute,  was  designated  a  national  laboratory  in 
1991  to  expand  federal  energy  research  and  development  (r&d)  capability 
in  alternative  energy  sources,  and  it  was  renamed  the  National  Renewable 
Energy  Laboratory  (Colo.).3  As  a  group,  the  10  laboratories  are  known  as 
the  national  laboratories. 

As  the  laboratories’  experience  and  research  capability  evolved,  mission 
emphases  shifted  among  them.  Sandia,  Los  Alamos,  and  Lawrence 
Livermore  acquired  primary  responsibility  for  nuclear  weapons  research 
and  development  and  the  largest  share  of  the  laboratories’  funds. 
Responsibility  for  research  in  the  environmental  and  biological,  energy, 
and  national  security  areas  was  distributed  among  all  10  laboratories  to 
varying  degrees.4  However,  the  Congress  and  doe  are  reassessing  this 
mission  configuration. 

Since  1980,  the  Congress  has  had  an  active  interest,  expressed  in  a  series 
of  laws,  in  seeing  that  more  of  the  national  laboratories’  outputs  be  put  to 
commercial  uses.5  Changing  needs  for  defense  technology  resulting  from 
the  end  of  the  cold  war  and  concern  with  maintaining  U.S  industry’s 
competitiveness  in  global  markets  have  led  several  members  of  Congress 
to  open  a  public  debate  and  propose  new  legislation  that  addresses  the 


2The  Atomic  Energy  Act  of  1946  delegated  responsibility  for  nuclear  weapons  research,  development, 
and  manufacture  as  well  as  civilian  uses  of  nuclear  energy  to  a  civilian  agency,  the  Atomic  Energy 
Commission.  Historically,  civilian  agencies  have  held  this  responsibility  in  the  United  States  and  most 
other  countries. 

3The  National  Renewable  Energy  Laboratory  was  established  in  1977  as  the  Solar  Energy  Research 
Institute.  One  of  DOE’s  program-directed  laboratories,  its  R&D  activities  focus  on  developing 
competitive  renewable  energy  and  related  technologies  for  the  nation  and  on  facilitating  their 
commercialization. 

4A  brief  description  of  the  10  national  laboratories’  missions  is  provided  in  appendix  I. 

5See  Technology  Transfer  Implementation  Status  of  the  Federal  Technology  Transfer  Act  of  1986 
(GAO/RCED-89-154,  May  30,  1989),  Implementation  of  the  Technology  Transfer  Act:  A  Preliminary 
Assessment  (GAO/T-PEMD-904,  May  3, 1990),  and  Diffusing  Innovations:  Implementing  the 
Technology  Transfer  Act  of  1986  (GAO/PEMD-91-23,  May  29,  1991)  for  a  discussion  of  congressional 
interest  in  technology  transfer  from  the  national  laboratories,  as  expressed  in  the  Stevenson-Wydler 
Technology  Innovation  Act  of  1980  (P.L.  96-480)  and  the  Federal  Technology  Transfer  Act  of  1986  (P.L 
99-502).  The  implementation  of  Executive  Order  No.  12591,  “Facilitating  Access  to  Science  and 
Technology,”  which  ordered  executive  departments  and  agencies  to  facilitate  collaboration  between 
federal  laboratories  and  other  public  and  private  sector  organizations,  also  is  considered  in  Diffusing 
Innovations. 

Note  that  this  legislation  and  our  reports  concern  all  laboratories  owned,  leased,  or  otherwise  used  by 
a  federal  agency.  The  10  national  laboratories  are  a  subset  of  this  population,  which  covers  at  least  10 
executive  branch  departments  and  297  laboratories.  See  also  the  National  Competitiveness 
Technology  Transfer  Act  (P.L  101-189),  which  in  1989  authorized  DOE  to  establish  technology  transfer 
as  a  mission  of  government-owned,  contractor-operated  laboratories,  such  as  the  national  laboratories, 
and  to  approve  formation  of  cooperative  research  and  development  agreements  (CRADAs)  between 
these  laboratories  and  industry. 
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national  laboratories’  missions,  structure,  and  cooperation  with  industry.6 
Among  the  alternatives  being  considered  in  the  public  debate  are  reducing 
all  the  laboratories’  budgets,  consolidating  or  closing  some  of  them,  and 
redirecting  their  weapons  development  mission  toward  commercial 
product-related  r&d  in  such  areas  as  technology  development  for 
environmental  restoration,  energy,  and  high-performance  computing. 

Underlying  these  discussions  are  questions  about  the  type  of  r&d  activities 
the  national  laboratories  are  performing  now,  the  nature  and  scope  of 
their  outputs,  and  their  potential  for  assisting  industry  in  bringing 
technology  to  the  marketplace.  This  report  is  an  effort  to  inform  the 
debate  by  providing  an  empirical  base  for  these  questions,  as  a  starting 
point  for  addressing  the  broader  issues.  It  examines  whether  the  balance 
of  laboratories’  effort  is  in  basic  and  applied  research  or  research  related 
to  commercial  product  development,  the  distribution  of  the  laboratories’ 
research  outputs,  and  their  potential  for  commercial  application.  Findings 
were  based  on  a  cross-section  of  the  laboratories’  r&d  activities  for  the 
period  1989-92.  However,  the  objectives  for  most  of  the  programs  in  the 
study  population  were  initiated  before  the  national  laboratories’  legislative 
mandate  for  technology  transfer  in  the  National  Competitiveness 
Technology  Transfer  Act  took  effect  in  late  1989.  In  most  fields  of  r&d, 
more  than  4  years  are  required  for  outputs  to  evolve  after  objectives  have 
been  established.  Therefore,  the  commercial  product-related  effort  we 
found  is  to  be  considered  a  baseline  against  which  future  activities  and 
outputs  can  be  measured. 


Methodology 


We  began  our  work  by  developing  a  comprehensive  description  of  current 
research  activities  in  the  10  laboratories.  We  chose  to  survey  the 
laboratories  directly  because  we  could  find  no  sufficiently  comprehensive 
existing  documentation.  We  collected  our  data  through  a  survey  of  the  10 
laboratories’  research  programs  and  the  facilities  and  equipment  that 
support  them.  The  survey  scope  consisted  of  all  major  research  programs 
and  facilities  with  costs  of  at  least  $10  million,  as  well  as  special 


6See  U.S.  Congress,  “Department  of  Energy  Laboratory  Technology  Act  of  1993,”  H.R.  1432,  sections  2, 
4,  5,  and  9. 
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nominations  by  the  laboratories  themselves  of  other  less  costly  programs 
and  facilities.7 

These  two  criteria  were  designed  to  ensure  that  all  large  subprograms  and 
smaller  subprograms  that  were  important  to  the  laboratories’  missions 
would  be  included  in  our  sample.  This  allowed  us  to  describe  the 
laboratories’  nugor  research  efforts.  However,  findings  based  on  these 
criteria  should  not  be  considered  representative  of  a  laboratory’s  entire 
research  effort  since  the  proportion  of  programs  budgeted  at  less  than 
$10  million  can  vary  from  one  laboratory  to  another. 

doe’s  Budget  and  Reporting  System  categories  provided  a  common 
classification  scheme  for  the  laboratories’  12  research  programs,  which 
permitted  cross-laboratory  comparisons  of  program  characteristics. 
Research  program  and  subprogram  names  are  shown  in  table  1.  The  data 
we  collected  on  these  programs  covered  fiscal  years  1989-92. 

We  conducted  pilot  tests  of  the  survey  methodology  and  data  collection 
instruments  at  Brookhaven  and  the  National  Renewable  Energy 
Laboratory.  We  then  revised  the  instrument  and  administered  one  version 
to  the  remaining  eight  laboratories.8  After  we  processed  the  survey 
responses,  we  asked  each  laboratory  to  confirm  by  letter  that  our  list  of 
research  programs  and  facilities  was,  in  fact,  complete. 


7A  research  program  in  the  survey  population  is  one  of  several  broad  areas  of  research  activity  taking 
place  within  a  laboratory  that  had  a  total  annual  budget  equal  to  or  exceeding  $10  million,  was  funded 
in  fiscal  year  1992,  and  was  planned  to  continue  in  fiscal  year  1993.  A  program  that  did  not  meet  these 
criteria  could  be  nominated  by  the  laboratory  for  inclusion  in  the  survey  on  the  basis  of  its  uniqueness 
or  contribution  to  science  and  technology  development.  Each  laboratory  could  nominate  up  to  10 
programs  in  this  category.  A  facility  in  the  survey  population  is  an  entity  that  houses  and  comprises 
the  equipment  used  in  conducting  R&D.  A  facility  could  be  a  building  or  defined  structure,  some  area 
within  a  structure,  or  a  defined  area  not  confined  to  a  structure  (for  example,  a  testing  area).  An 
acquisition  cost  of  $10  million  or  nomination  by  the  laboratory  for  its  uniqueness,  world-class  quality, 
synergistic  effect  when  combined  with  other  facilities,  or  contribution  to  a  given  research  program  or 
project  qualified  a  facility  for  inclusion  in  the  survey.  Each  laboratory  could  nominate  up  to  15 
facilities  that  did  not  meet  the  $10-million  acquisition  cost  criterion. 

8The  national  laboratory  inventory  data  collection  instrument  has  two  parts— part  I  concerns  the 
laboratory’s  research  programs;  part  II  requests  information  about  a  laboratory’s  major  research 
facilities.  This  report  contains  data  for  eight  of  the  10  laboratories  from  part  I  of  the  revised  data 
collection  instrument,  which  is  reproduced  in  appendix  II.  Data  reported  for  Brookhaven  and  the 
National  Renewable  Energy  Laboratoiy  were  collected  with  the  pilot-test  versions  of  the  data 
collection  instrument,  which  were  somewhat  different  from  the  instrument  in  appendix  II. 
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Table  1:  Classification  for  National 
Laboratory  Research  Programs 


Program 

Subprogram  category8 

Energy  Research 

Magnetic  fusion  (AT) 

High-energy  physics  (KA) 

Superconducting  supercollider  (KS) 

Nuclear  physics  (KB) 

Basic  energy  sciences  (KC) 

Biological  and  environmental  (KP) 

Conservation  and  Renewable 
Energy 

Electric  energy  systems  (AK) 

Geothermal  (AM) 

Solar  energy  (EB) 

Building  and  community  systems  (EC) 

Industrial  Energy  Conservation  (ED) 

Transportation  (EE) 

State  and  local  programs  (EF) 

Environment,  Safety  and 
Health 

Environment,  safety,  and  health  (HA) 

Nuclear  Energy 

Nuclear  energy  R&D  (AF) 

Uranium  enrichment  (CD) 

Defense  Programs 

Weapons  activities  (GB) 

Verification  and  control  technologies  (GC) 

Nuclear  safeguards  and  security  (GD) 

Production  and  surveillance  support  to  the  nuclear 
weapons  complex  (NM) 

New  Production  Reactors 

New  production  reactors  (NP) 

Environmental  Restoration 
and  Waste  Management 

Environmental  restoration  and  waste 
management — defense  (EW,  EM) 

Environmental  restoration  and  waste 
management — nondefense  (EX) 

Fossil  Energy 

Coal  (AA) 

Petroleum  (AC) 

Strategic  petroleum  reserve  (SA) 

Civilian  Radioactive  Waste 
Management 

Nuclear  waste  fund  (DB) 

Policy  Planning  and  Analysis 

Policy,  analysis,  and  systems  studies  (PE) 

Intelligence 

Intelligence  (NT) 

Work  for  others 

Work  for  others  (WFO) 

aDOE's  Budget  and  Reporting  System  subprogram  category  codes  are  in  parentheses.  For 
purposes  of  this  project,  we  have  used  the  code  “WFO”  to  identify  work  for  others  programs. 


Source:  National  laboratories'  institutional  plans  for  fiscal  years  1991-96. 


The  national  laboratories  engage  in  a  wide  range  of  defense  and 
nondefense  R&D-related  activities.  These  range  from  generating  hypotheses 
and  testing  fundamental  science  principles  to  assisting  a  potential  user  in 
adapting  laboratory  outputs  to  a  production  or  service  delivery  system.  To 
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analyze  the  extent  to  which  the  laboratories  are  engaged  in  basic  and 
applied  research  or  research  related  to  commercial  product  development, 
we  divided  their  activities  into  five  categories:  basic  research,  applied 
research,  development,  technology  transfer,  and  technical  assistance. 

Basic  research  is  research  undertaken  primarily  to  gain  fuller  knowledge 
or  understanding  of  a  subject  and  to  contribute  to  the  knowledge  base  in 
the  field  of  investigation.  Applied  research  is  research  directed  toward 
the  practical  use  of  knowledge  or  understanding  of  a  subject  to  meet  a 
recognized  need.  Development  is  research  directed  toward  the 
production  of  useful  materials,  devices,  systems,  or  methods,  including  the 
design  and  development  of  prototypes  or  processes.  Development  has 
some  type  of  product  as  the  output  goal,  but  may  conclude  with  a 
prototype  rather  than  a  usable  good.  Additional  time,  research,  and  testing 
are  usually  required  to  convert  the  prototype  to  a  weapon  or  commercially 
viable  product. 

Because  the  national  laboratories  perform  r&d  only  through  the 
development  stage,  additional  mechanisms  and  arrangements  are  required 
to  achieve  application  of  the  laboratories’  outputs  in  the  public  or  private 
sector.  These  activities  are  technology  transfer  and  technical  assistance. 
Technology  transfer  is  the  process  that  fosters  the  use  of  devices, 
processes,  “know-how,”  or  scientific  and  technical  information  produced 
in  a  national  laboratory  by  universities,  private  industry,  or  government 
agencies.  It  includes  making  potential  users  aware  of  the  laboratories’ 
research  outputs,  assisting  in  their  selection  or  use,  and  collaborating  with 
representatives  of  private  industry  and  public  or  nonprofit  institutions  to 
ensure  that  some  of  the  laboratories’  outputs  will  have  commercial  or 
public  applications.  Technical  assistance  applies  the  laboratory’s 
expertise  to  practical  problems  but  does  not  involve  the  use  of  a 
laboratory’s  outputs.  It  is  any  form  of  assistance,  other  than  financial,  to  a 
state  or  local  government  or  a  business,  including  publications, 
workshops,  conferences,  studies,  or  telephone  consultation. 

Development,  technical  assistance,  and  technology  transfer  are  the  three 
national  laboratory  research  activities  related  to  commercial  product 
development.  All  five  categories,  already  used  by  the  laboratories  but 
specially  grouped  for  our  analysis,  constitute  a  natural  framework  that, 
together  with  doe’s  program  classification  scheme,  allowed  us  to  look  at 
R&D-related  activity  across  all  10  laboratories,  using  expenditures  as  a 
measure  of  activity. 
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Table  2:  National  Laboratory  Budgets 
for  Fiscal  Year  1992 


Recognizing  that  the  laboratories  do  not  maintain  records  of  their  r&d 
expenditures  in  terms  of  our  five  categories,  we  asked  managers  of  the 
subprograms  in  our  study  population  to  estimate,  for  each  subprogram 
they  managed,  the  proportion  of  funds  expended  in  each  of  the  five  areas. 
Our  analysis  of  r&d  activity  is  therefore  presented  as  percentages,  not 
actual  dollar  values.  To  provide  a  context  for  considering  our  findings,  we 
present  in  table  2  the  fiscal  year  1992  budgets  for  subprograms  in  the  study 
population  that  were  included  in  our  analysis  and  the  laboratories’  total 
budgets  in  fiscal  year  1992. 


National  laboratory 

R&D  budget8 

Total  budgetb 

Argonnne 

$357.8 

$577.8 

Lawrence  Berkeley 

193.3 

270.6 

Oak  Ridge 

346.6 

726.9 

Pacific  Northwest 

340.4 

417.0 

Idaho  National  Engineering"1 

98.7 

931.5 

Lawrence  Livermore 

727.6 

1395.5 

Los  Alamos 

710.9 

1239.6 

Sandia 

722.1 

1389.6 

Brookhaven 

253.1 

472.6 

National  Renewable  Energy 

113.4 

131.2 

Total 

$3,863.9 

$7,552.3 

aln  millions  of  dollars;  includes  salaries  and  wages,  overhead,  expendables,  capital  equipment, 
and  other  factors  for  a  laboratory's  subprograms  in  the  study  population.  Construction  costs  are 
not  included.  Dollar  values  are  the  total  of  a  laboratory's  research  subprograms’  budgets  reported 
by  program  managers. 

bln  millions  of  dollars;  includes  salaries  and  wages,  expendables,  overhead,  capital  equipment, 
general  purpose  equipment,  construction,  and  all  other  factors  for  all  of  a  laboratory’s  research, 
educational,  administrative,  and  other  activities.  Dollar  values  are  the  estimated  budget 
authorization  reported  in  the  laboratory  5-year  institutional  plan. 

cldaho  National  Engineering  Laboratory  conducts  energy,  defense,  environmental,  and  nuclear 
reactor  research;  provides  scientific  training  in  nuclear  reactors  and  waste  management;  and 
designs,  constructs,  and  operates  defense  production  facilities.  Idaho  laboratory  representatives 
determined  that  much  of  their  work  in  the  environmental,  nuclear  energy,  and  defense  areas  did 
not  meet  criteria  for  inclusion  in  the  inventory.  In  addition,  the  total  laboratory  budget  estimate  in 
the  fiscal  year  1992  institutional  plan  included  $250  million  for  construction,  which  was  not 
included  in  the  R&D  budget  for  the  study  population. 


We  also  examined  the  laboratories’  outputs.  As  output  measures,  we 
selected  products  of  laboratory  r&d  that  were  clearly  identifiable  to  our 
respondents  and  for  which  they  were  likely  to  maintain  records.  Since  our 
study  objective  was  to  examine  the  balance  among  laboratory  activities 
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rather  than  their  impact,  we  focused  on  outputs  of  r&d  activity  that 
occurred  within  the  laboratories  rather  than  their  efforts  at  job  creation  or 
increased  sales.  Because  of  great  variation  in  the  size,  scope,  field  of 
investigation  and  funding  level  of  the  subprograms  in  the  study  population, 
both  within  and  among  laboratories,  we  presented  our  findings  as  simple 
tabulations,  rather  than  as  standardized  units.  Use  of  a  single  measure  for 
standardizing  the  outputs,  such  as  dollar  of  funding  per  output,  would 
have  failed  to  account  for  variations  among  the  subprograms  on  other 
dimensions.  Moreover,  because  of  the  institutional  complexity  this 
variation  represents,  we  interpreted  our  output  findings  very 
conservatively,  treating  them  as  measures  of  activity  rather  than  indicators 
of  performance.  We  looked  at  the  outputs  in  two  broad  categories: 

(1)  publications  and  reports  and  (2)  outputs  related  to  commercial 
product  development.  The  outputs  attributed  to  each  category  are 
described  in  the  Principal  Findings  section. 

Finally,  we  looked  at  three  other  indicators— the  formation  of  cooperative 
r&d  agreements,  r&d  effort  devoted  to  critical  technologies,  and  program 
managers’  assessment  of  their  on-going  research — to  gauge  the 
laboratories’  potential  for  commercial  product  development. 

We  conducted  our  review  in  accordance  with  generally  accepted 
government  auditing  standards.9 


Results  in  Brief 


The  national  laboratories  devoted  slightly  more  than  half  (52.4  percent)  of 
their  r&d  funds  to  research  related  to  commercial  product  development 
during  fiscal  year  1992.  This  includes  30.9  percent  for  development, 

14.4  percent  for  technical  assistance,  and  7  percent  for  technology 
transfer.  However,  most  of  the  10  laboratories’  development  work 
(56.7  percent)  was  devoted  to  defense  (which  may  have  more  limited 
market  opportunities)  rather  than  nondefense  research.  Less  than  half 
(44.6  percent)  of  the  laboratories’  effort  was  spent  on  basic  and  applied 
research:  17.4  percent  on  basic  research  and  27.2  percent  on  applied 
research. 

The  10  laboratories  produced  many  more  publications  and  reports 
(21,593)  than  they  did  outputs  related  to  commercial  product  development 
(2,510)  in  fiscal  year  1992.  We  expected  this  finding  because  publications 
and  reports  are  the  primary  mechanism  for  disseminating  the  results  of  all 
types  of  R&D-related  activities.  Further,  we  found  that  the  defense  program 


9See  appendix  III  for  a  more  detailed  discussion  of  survey  methodology. 
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supported  most  of  the  commercial  product-related  outputs,  and  that 
Sandia,  Los  Alamos,  and  Lawrence  Livermore — which  are  known  as 
weapons  laboratories — produced  most  of  these  outputs.  However, 
because  we  asked  the  laboratories  to  classify  these  outputs — prototype 
devices  and  materials,  algorithms,  software,  and  other  commercial 
products  or  processes  that  have  an  identified  commercial  use — at  a  time 
that  is  still  several  years  away  from  market  entry,  whether  or  to  what 
degree  they  will  actually  achieve  commercial  application  is  unknown. 

With  regard  to  the  laboratories’  potential  for  commercial  product 
development,  we  found  that  such  potential  exists,  based  on  the  three 
indicators  we  examined.  Activity  is  increasing  in  the  formation  of 
cooperative  r&d  agreements  between  the  laboratories  and  industry.  These 
agreements  increased  from  17  in  fiscal  year  1989  to  196  in  fiscal  year  1992. 
Of  course,  they  ensure  only  that  the  laboratories  and  industry  will 
collaborate  on  r&d;  a  commercial  innovation  may  or  may  not  be  produced. 
With  respect  to  the  research  emphasis  selected,  about  three-fourths 
(74.1  percent)  of  the  10  laboratories’  r&d  expenditures  were  focused  on 
those  technologies  the  National  Critical  Technologies  Panel  had  identified 
as  vital  to  national  needs.10  Here  again,  the  potential  for  commercial 
product  development  exists,  but  the  actual  outcome  will  not  be  known  for 
several  years.  Finally,  over  half  (67.6  percent)  of  the  managers  of  programs 
with  commercial  product  potential  expected  clear  evidence  of  that 
potential  to  emerge  within  5  years  or  less  from  fiscal  year  1992. 


Principal  Findings 


Balance  Among  To  examine  the  balance  of  the  national  laboratories’  current  R&D-related 

R&D-Related  Activities  activities,  we  analyzed  the  distribution  of  laboratory  expenditures  for  r&d 

within  and  among  laboratories  and  research  programs.  For  the  10 
laboratories  overall,  R&D-related  activity  was  almost  evenly  divided 
between  basic  and  applied  research  on  the  one  hand,  and  research  related 
to  commercial  product  development  on  the  other.  Approximately 
8  percent  more  of  the  effort  was  devoted  to  r&d  activities  related  to 
commercial  product  development,  as  shown  in  figure  1.  More  applied 


10The  Congress  established  the  National  Critical  Technologies  Panel  through  the  Fiscal  Year  1990 
Defense  Authorization  Act  (P.L.  101-189),  an  amendment  to  the  National  Science  and  Technology 
Policy,  Organization,  and  Priorities  Act  of  1976.  Congress  asked  the  Panel  to  identify  up  to  30  areas  of 
technological  development  they  consider  essential  for  the  nation’s  long-term  security  and  economic 
prosperity.  The  Panel  reports  to  the  Congress  and  the  President  on  the  critical  technologies  biennially 
through  the  year  2000. 
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research  than  basic  research  was  conducted:  27.2  percent  versus 
17.4  percent.  Among  research  activities  related  to  commercial  product 
development,  most  (30.9  percent)  was  development,  but  more  activity  was 
devoted  to  technical  assistance  (14.4  percent)  than  technology  transfer 
(7  percent).  Thus,  R&D-related  activity  directly  targeted  on  potential 
commercial  applications  of  the  laboratories’  outputs  currently  constitutes 
the  smallest  proportion  of  the  laboratories’  R&D-related  effort.  Despite  its 
small  size,  however,  this  level  of  effort  exceeds  the  laboratories’  minimum 
statutory  requirement  for  technology  transfer  activity.11 


Figure  1 :  National  Laboratory  Mean 
Percent  Expenditures  for  R&D-Related 
Activities8 


3.0% 

Otherb 


Research  related  to  commercial 
product  development0 


Basic  and  applied  research 


aFiscal  year  1992. 

bSubprogram  expenditures  for  activities  other  than  basic  and  applied  research  or  research 
related  to  commercial  product  development,  such  as  training  graduate  students  and  postdoctoral 
fellows  or  safety  procedures. 

development,  technical  assistance,  and  technology  transfer. 


1  ‘The  Stevenson-Wydler  Act  requires  that  each  federal  agency  with  one  laboratory  or  more  make 
available  at  least  one-half  of  1  percent  of  its  R&D  budget  for  technology  transfer  activities  and  that 
laboratories  having  200  or  more  scientific,  engineering,  and  technical  full-time-equivalent  (FTE)  staff 
assign  at  least  one  FTE  to  an  Office  of  Research  and  Technology  Applications,  which  has  formal 
responsibility  for  the  laboratory’s  technology  transfer  activities. 
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These  overall  percentages,  however,  mask  major  differences  among  the 
laboratories  with  regard  to  r&d  funding  distribution.  (See  table  IV.  1  in 
appendix  IV.)  Four  laboratories — Argonne,  Lawrence  Berkeley,  Oak  Ridge, 
and  Brookhaven — spent  25  percent  or  more  of  their  research  funds  on 
basic  research.  These  laboratories  account  for  over  half  (59.3  percent)  of 
the  total  national  laboratory  research  budget  that  is  spent  on  basic 
research.  (See  table  IV.2.)  Los  Alamos  spent  19.4  percent  of  its  r&d  funds 
to  support  its  mission  to  perform  “basic  research  in  selected  disciplines 
that  help  maintain  an  outstanding  science  and  technology  base.”  Only 
about  10  percent  or  less  of  the  laboratory  research  budget  was  spent  on 
basic  research  at  the  other  laboratories.  The  energy  research  program 
accounted  for  the  greatest  proportion  of  funds  spent  on  basic  research, 
both  within  and  among  research  program  areas.  (See  tables  IV.3  and  IV. 4.) 

As  table  IV.  1  shows,  four  laboratories — Oak  Ridge,  Pacific  Northwest, 
Lawrence  Livermore,  and  Los  Alamos — spent  29  percent  or  more  of  their 
research  funds  on  applied  research.  Among  the  10  laboratories,  Lawrence 
Livermore  and  Los  Alamos  accounted  for  almost  half  (47.9  percent)  of 
applied  research  expenditures.  (See  table  IV.2.)  Most  applied  research  was 
supported  by  programs  in  the  areas  of  defense,  energy  research,  and  work 
for  others.  (See  table  IV.4.) 

As  noted  earlier,  most  of  the  laboratories’  development  work,  the  most 
product-oriented  of  r&d  activities,  was  devoted  to  defense,  rather  than, 
nondefense,  research.  Almost  three-quarters  (71.5  percent)  of  all  the 
laboratories’  development  research  was  conducted  at  Lawrence 
Livermore,  Los  Alamos,  and  Sandia  (See  table  IV.2.)12  In  turn,  the  largest 
share  of  development  research  was  performed  in  the  defense  and  nuclear 
energy  programs.  (See  tables  IV.3  and  IV.4.)  Therefore,  while  it  is  true  that 
across  the  10  laboratories,  a  greater  proportion  of  research  funding  was 
devoted  to  activities  more  closely  related  to  commercial  product 
development  than  to  basic  and  applied  research,  most  of  these  funds 
currently  support  defense  research.  To  determine  whether  this  research 
will  have  commercial  opportunities  for  use,  we  examined  the  national 
laboratories’  outputs. 


12Pacific  Northwest  and  the  National  Renewable  Energy  Laboratory  each  spend  almost  one-third  of 
their  research  funds  on  development  However,  these  laboratories  are  funded  at  a  substantially  lower 
level  than  the  weapons  laboratories. 
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National  Laboratories’ 
R&D  Outputs 


A  second  measure  of  the  type  of  effort  in  which  the  national  laboratories 
are  engaged— as  between  basic  and  applied  research  or  research  related 
to  commercial  product  development — is  output.  The  laboratories  produce 
two  major  types  of  outputs:  (1)  publications  and  reports,  and  (2)  outputs 
related  to  commercial  product  development.13  Table  3  shows  that,  across  a 
4-year  period,  most  of  the  laboratories’  outputs  were  publications  and 
reports.  This  finding  was  expected  because  reports  and  publications  are 
the  primary  mechanisms  for  diffusion  of  R&D  findings,  and  they  are 
prepared  at  all  stages  of  the  r&d  process.  Reports,  conference  papers,  and 
published  articles,  which  can  be  produced  more  quickly  than  books  and 
book  chapters,  substantially  outnumber  the  latter. 

As  we  discussed  above,  a  slightly  higher  percentage  of  the  laboratories’ 
expenditures  was  devoted  to  r&d  activities  related  to  commercial  product 
development  than  to  basic  and  applied  research;  nevertheless,  few  of  their 
outputs  were  commercial  product-related.14  Prototype  devices  and 
materials,  algorithms,  and  software  are  the  largest  number  of  outputs  in 
this  group.  These  outputs  tend  to  arise  from  the  development  stage  of  the 
r&d  process,  which  often  occurs  several  years  before  production  of  a 
marketable  or  usable  good.  Not  all  outputs  of  the  development  stage  will, 
of  course,  achieve  commercial  application. 

Most  of  the  prototype  devices  and  materials,  algorithms,  and  software,  as 
indicated  in  tables  V.l  and  V.2  in  appendix  V,  were  produced  at  the 
weapons  laboratories,  and  most  were  funded  by  doe’s  defense  program. 
Other  outputs  laboratory  managers  identified  as  commercial  products  or 
commercial  processes  also  tend  to  arise  from  the  development  stage. 
Although  they  will  require  a  substantial  additional  investment  before  they 
are  ready  to  market,  these  products  or  processes  will  more  likely  result  in 
actual  commercial  applications  because  a  potential  commercial  use  has 
already  been  identified.  Most  of  these  outputs  were  produced  by  Los 
Alamos,  Sandia,  and  Pacific  Northwest,  and  the  defense  program  supports 
most  of  the  research  that  has  led  to  these  outputs.  The  point  here  is  that 
although  defense-funded  r&d  has  produced  more  outputs  that  could  lead 
to  commercial  products,  whether  these  outputs  will  achieve  commercial 
application  is  still  unknown. 


13A  third  set  of  outputs,  which  are  not  related  to  these  categories,  is  designated  as  “other.” 

14Prototype  devices  and  materials,  algorithms,  software,  patents,  licenses,  commercial  products, 
commercial  processes,  and  spin-off  companies  are  defined  as  laboratory  outputs  related  to 
commercial  product  development.  The  research  program  managers  have  identified  a  potential 
commercial  use  for  commercial  products  and  commercial  processes,  but  these  outputs  have  achieved 
at  most  a  precompetitive  stage  of  development.  We  view  patents,  licenses,  and  spin-off  companies  as 
evidence  of  intent  to  pursue  a  marketing  strategy  for  a  research  output 
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Table  3:  National  Laboratories’  Research  Program  Outputs  for  Fiscal  Years  1989-92 


1992 

1991 

1990 

1989 

Total 

Outputs 

All  4 
years® 

Any  4 
yearsb 

All  4 
years® 

Any  4 
years'5 

All  4 
years® 

Any  4 
years'5 

All  4 
years® 

Any  4 
years'5 

All  4 
years® 

Any  4 
years'5 

Publications  and  reports 

Books 

88 

94 

120 

121 

181 

181 

120 

120 

509 

516 

Articles 

6,153 

6,471 

6,330 

6,387 

5,612 

5,652 

5,959 

5,970 

24,054 

24,480 

Book  chapters0 

355 

371 

269 

271 

321 

322 

280 

280 

1,225 

1,244 

Reports 

6,245 

6,802 

5,416 

5,542 

4,761 

4,851 

4,497 

4,533 

20,919 

21,728 

Conference  papers 

7,237 

7,855 

6,812 

6,931 

6,498 

6,572 

6,421 

6,449 

26,968 

27,807 

Subtotal 

20,078 

21,593 

18,947 

19,252 

17,373 

17,578 

17,277 

17,352 

73,675 

75,775 

Outputs  related  to  commercial 

product  development 

Prototypes 

501 

566 

485 

490 

497 

500 

492 

492 

1,975 

2,048 

Algorithms 

732 

778 

713 

715 

632 

634 

552 

552 

2,629 

2,679 

Software 

508 

554 

417 

422 

350 

381 

314 

315 

1,589 

1,669 

Patents0 

159 

237 

139 

199 

145 

203 

148 

166 

591 

805 

Licenses 

210 

303 

128 

149 

42 

60 

39 

47 

419 

559 

Commercial  products 

28 

36 

17 

24 

19 

26 

17 

21 

81 

107 

Commercial  processes 

22 

29 

13 

20 

11 

18 

14 

18 

60 

86 

Spin-off  company0 

6 

7 

2 

3 

8 

11 

11 

12 

27 

33 

Subtotal 

2,166 

2,510 

1,914 

2,022 

1,704 

1,833 

1,587 

1,623 

7,371 

7,988 

Other 

New  programs0 

153 

159 

142 

146 

137 

139 

98 

100 

530 

544 

Invention  disclosures0 

320 

617 

336 

618 

247 

481 

232 

341 

1,135 

2,057 

Otherd 

510 

523 

472 

483 

412 

421 

404 

414 

1,798 

1,841 

Subtotal 

983 

1,299 

950 

1,247 

796 

1,041 

734 

3,463 

4,442 

“These  are  outputs  for  subprograms  that  were  in  operation  every  year  during  fiscal  years 
1989-92. 


bThese  are  outputs  for  subprograms  that  were  initiated  in  any  year  during  fiscal  years  1989-92. 

“Responses  were  not  collected  from  Brookhaven  National  Laboratory  and  National  Renewable 
Energy  Laboratory. 

“Research  subprogram  outputs  other  than  those  listed  above,  such  as  technical  abstracts, 
workshops  for  laboratory  users,  and  an  electronic  bulletin  board  service. 


Patent  applications  may  be  submitted  for  inventions  throughout  the  entire 
r&d  process,  but  a  license  is  usually  acquired  only  when  a  decision  to 
market  a  technology  has  been  made.  The  number  of  licenses  awarded, 
therefore,  is  a  stronger  measure  of  output  activity  related  to  commercial 
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product  development  than  the  number  of  patents.  A  trend  in  the  data 
indicative  of  the  laboratories’  production  of  outputs  related  to  commercial 
product  development  is  the  increase  in  the  number  of  licenses  awarded 
dining  fiscal  years  1989  through  1992.  (See  table  3.)  In  fiscal  year  1992, 
Sandia  and  Pacific  Northwest  awarded  the  most  licenses,  and  most 
licensed  outputs  were  supported  by  defense  program  research.  (See  tables 
V.l  and  V.2.) 

We  expected  to  find  that  most  commercial  product-related  outputs  were 
supported  by  research  programs  that  spent  most  of  their  r&d  funds  for 
development.  However,  the  r&d  expenditures  of  those  programs  that 
supported  the  most  outputs  related  to  commercial  product  development 
covered  the  range  of  r&d  activities.  We  found  that  in  fiscal  year  1992,  four 
research  programs — energy  research,  conservation  and  renewable  energy, 
defense,  and  work  for  others — supported  most  of  the  commercial 
product-related  outputs  of  all  types  and  that,  over  4  years,  commercial 
product-related  output  production  had  been  increasing  each  year  in  three 
of  the  programs,  as  shown  in  figure  V.  1.  We  also  found  that  in  fiscal  year 
1992,  the  largest  proportion  of  expenditures  in  the  defense  and 
conservation  and  renewable  energy  programs  was  for  development. 

As  expected,  the  defense  and  conservation  and  renewable  energy 
programs  supported  more  of  the  outputs  specifically  designated  as 
commercial  products  and  processes  than  any  of  the  10  other  research 
programs.  However,  in  looking  more  closely  at  these  four  programs,  we 
found  some  interesting  differences.  Work  for  others,  which  supports  more 
commercial  product  and  process-type  outputs  than  eight  other  programs, 
devoted  a  slightly  higher  proportion  of  r&d  expenditures  to  applied 
research  than  to  development.  But  in  energy  research,  which  supports 
more  commercial  product-  and  process-related  outputs  than  nine  other 
programs,  the  largest  proportion  of  expenditures  was  for  basic  research. 
(See  table  V.3.) 


Laboratories’  Potential  for 
Commercial  Product 
Development 


We  looked  at  three  indicators  of  the  national  laboratories’  potential  for 
commercial  product  development:  (1)  formation  of  cooperative  research 
and  development  agreements;  (2)  proportion  of  r&d  expenditures  in 
critical  technology  areas;  and  (3)  research  program  managers’ judgments 
about  their  programs’  outputs.  Of  these  three,  the  most  frequently  used 
indicator  of  the  national  laboratories’  potential  for  commercial  product 
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development  is  the  formation  of  cradas.15  Here  we  found  a  major 
increase  in  activity.  The  national  laboratories  reported  that  from  fiscal 
year  1989  through  1992,  they  entered  into  196  cradas.  Among  programs  in 
the  study  population  in  operation  all  4  years,  the  number  of  new  cradas 
formed  increased  from  17  in  fiscal  1989  to  130  in  fiscal  1992. 

Sandia  and  Oak  Ridge  laboratories  were  most  active  in  entering  into 
cradas.  (See  table  VI.  1.  in  appendix  VI.)  Most  were  formed  for  research 
sponsored  by  programs  in  the  defense  and  conservation  and  renewable 
energy  areas.  (See  table  VL2.)  The  greatest  increase  in  crada  formation 
occurred  at  Sandia,  where  74  cradas  were  in  effect  in  fiscal  year  1992. 
Fifty-three  of  the  cradas  effective  in  fiscal  year  1992  were  sponsored  by 
the  defense  program  technology  transfer  initiative  at  Sandia.  This 
subprogram  was  initiated  in  June  1990  to  identify  opportunities  for 
commercializing  technologies  produced  by  DOE-funded  defense  research 
activities  in  such  areas  as  advanced  manufacturing  and  precision 
engineering,  materials  and  processes,  advanced  microelectronics  and 
photonics,  and  computer  architecture  and  applications. 

Although  the  national  laboratories  do  not  yet  have  a  legislative  mandate  or 
mission  for  research  in  the  critical  technologies,  their  research  program 
managers  reported  that  74.1  percent  of  r&d  expenditures  are  devoted  to 
work  in  critical  technology  areas.  This  research  was  distributed  over  the 
22  areas  identified  by  the  National  Critical  Technologies  Panel,  with  the 
greatest  concentration  in  energy  technologies  (13.6  percent);  pollution 
minimization,  remediation,  and  waste  management  (8.8  percent); 
computer  simulation  and  modeling  (6.7  percent);  and  materials  synthesis 
and  processing  (6.2  percent).  (See  table  VI.3.) 

Work  in  these  critical  technology  areas  was  distributed  broadly  among  the 
laboratories  and  research  programs.  Five  laboratories — Argonne, 
Lawrence  Berkeley,  Oak  Ridge,  Idaho,  and  Lawrence  Livermore — devoted 
approximately  20-30  percent  of  their  research  funds  to  energy 
technologies.  (See  table  VI.3.)  Pacific  Northwest  expended  the  greatest 
proportion  of  r&d  funds  (41.3  percent)  on  pollution  minimization 


1BA  CRADA  is  a  contractual  provision  created  to  foster  technology  transfer  from  federal  laboratories  to 
the  private  sector.  Agreements  can  be  formed  with  businesses  as  well  as  nonprofit  organizations  and 
state  and  local  government  agencies.  The  National  Competitiveness  Technology  Transfer  Act  of  1989 
authorized  government-owned,  contractor-operated  laboratories,  such  as  the  national  laboratories,  to 
enter  into  CRADAs. 

CRADA  formation  represents  commercial  product  potential  because  it  establishes  the  process 
uniquely  for  conducting  research  related  to  commercial  product  development.  The  conclusion  of  the 
period  of  performance  does  not  guarantee  that  the  research  will  have  been  completed  or  that  a 
market-ready  product  will  have  been  developed. 
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technologies.  Idaho  and  Lawrence  Livermore  were  most  active  in 
computer  simulation  and  modeling.  Oak  Ridge  and  Los  Alamos  devoted 
the  greatest  percentage  of  effort  to  materials  synthesis  and  processing.  As 
a  group,  the  laboratories  devoted  approximately  three-fourths  of  their  r&d 
expenditures  to  research  in  critical  technology  areas,  but  Sandia  and  Los 
Alamos  expended  only  about  half  of  their  resources  on  critical 
technologies  research.  All  of  the  research  programs  sponsored  research  in 
critical  technologies  to  some  degree,  with  the  least  effort  expended  by 
environment,  safety,  and  health.  (See  table  VIA) 

Finally,  laboratory  research  program  managers’  judgments  about  their 
research  programs’  potential  for  commercial  product  development 
were  optimistic.  Among  the  subset  of  all  national  laboratory  programs 
with  a  potential  for  commercial  product  development,  almost  58  percent 
of  the  program  managers  expected  that  development  to  occur  within  5 
years  of  fiscal  1992.  (See  figure  VI.  1.)  An  additional  27.6  percent  reported 
that  their  program  has  the  potential  for  commercial  product  development 
within  5-10  years. 


Conclusions 


As  of  1992,  the  national  laboratories  spent  slightly  more  than  half  of  their 
r&d  funds  on  research  related  to  commercial  product  development. 
However,  most  of  this  r&d  was  performed  at  the  weapons  laboratories  and 
was  supported  by  the  defense  and  nuclear  energy  programs.  Analysis  of 
the  outputs  produced  by  the  national  laboratories  indicated  that 
defense-funded  research  produced  more  outputs — prototype  devices  and 
materials,  algorithms,  software,  and  other  products  and  processes  that 
have  an  identified  commercial  application — that  are  precursors  to 
marketable  goods,  but  at  this  point,  whether  they  will  achieve  commercial 
application  is  not  known. 

Moreover,  three  indicators  of  the  laboratories’  potential  for  commercial 
product  development — crada  formation,  critical  technology  research,  and 
program  managers’  expectations  for  commercial  potential — showed  that 
some  activity  was  occurring,  crada  formation  was  increasing,  but  these 
arrangements  ensure  only  that  collaboration  between  the  laboratories  and 
industry  will  occur,  not  that  a  commercial  product  will  be  generated. 
Almost  three-fourths  of  the  laboratories’  effort  was  devoted  to  research  in 
critical  technology  areas,  but  achievement  of  commercial  application  will 
not  be  known  for  several  years.  Over  half  of  the  managers  of  research 
subprograms  that  have  commercial  product  potential  expected 
innovations  to  arise  within  5  years,  but  these  expectations  must  be 
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considered  “best  educated  guesses.”  While  we  can  conclude,  therefore, 
that  the  national  laboratories’  were  engaged  in  slightly  more  research 
related  to  commercial  product  development  than  basic  and  applied 
research,  it  is  too  early  to  determine  whether  this  activity  will  produce 
technologies  with  commercial  uses. 


We  requested  comments  on  a  draft  report  and  received  a  response  from 
doe  and  the  10  national  laboratories,  doe  questioned  the  definitions  and 
categories  we  defined  to  analyze  the  laboratories’  R&D-related  activities 
and  our  finding  that  the  laboratories  perform  slightly  more  research 
related  to  commercial  product  development  than  basic  and  applied 
research,  doe  also  thought  that  this  study  should  have  examined  additional 
institutional  factors,  including  the  r&d  activities  of  other  agencies,  and 
should  have  used  data  maintained  by  doe  headquarters  rather  than 
surveyed  the  laboratories  for  data. 

We  note  that  the  definitions  for  R&D-related  activities  we  employed  are 
derived  from  a  Congressional  Budget  Office  study  of  the  federal  r&d 
enterprise,  our  study  of  the  Technology  Transfer  Act  of  1986,  and  expert 
opinion.  We  also  disagree  with  doe’s  proposed  broader  scope  for  this 
study  because  it  exceeds  our  study  objective  and  would  have  required 
additional  data  collection  and  analyses  that  are  beyond  the  study  scope. 
Furthermore,  our  exploration  of  data  available  at  doe  headquarters  found 
that  it  was  not  adequate  to  satisfy  our  information  needs. 

Eight  laboratories  agreed  with  the  report’s  objective,  analyses,  and 
conclusions.  However,  one  of  this  group,  Lawrence  Berkeley,  thought  that 
the  relationship  of  commercial  product  development  to  the  broader  needs 
of  industry  and  the  nation  should  have  been  addressed  in  the  study.  Two 
of  the  laboratories  raised  issues  about  study  methodology.  Idaho  believed 
that  a  greater  proportion  of  the  budget  for  its  subprograms  should  have 
been  included  in  the  study  sample.  Oak  Ridge  questioned  the  effect  of  the 
study’s  sampling  methodology  on  output  findings  for  the  laboratory  and 
the  definition  of  the  category  called  outputs  related  to  commercial  product 
development. 

Lawrence  Berkeley  said  that  we  had  overlooked  an  important  issue.  The 
laboratory  thought  that  the  study  should  have  included  an  examination  of 
the  relationship  of  the  national  laboratories’  role  in  commercial  product 
development  to  the  broader  needs  of  industry  and  the  nation.  We  agree 
that  this  issue  is  important  to  address  as  part  of  the  public  debate  about 
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the  laboratories’  missions  and  structure.  However,  we  disagree  that  it 
should  have  been  examined  in  this  report,  which  focuses  on  establishing 
an  empirical  baseline  of  national  laboratories’  activities. 

doe’s  Idaho  Operations  Office  responded  for  Idaho  National  Engineering 
Laboratory.  The  Idaho  Operations  Office  said  that  the  budget  figure 
reported  for  Idaho  subprograms  included  in  the  study  sample  should  have 
been  higher.  We  did  not  agree  to  revise  Idaho’s  budget  figure,  because  to 
do  so  would  have  violated  the  study  methodology  used  to  sample 
programs  at  other  laboratories. 

Oak  Ridge  took  the  position  that  most  of  its  commercial  product-related 
outputs  were  produced  by  subprograms  that  were  not  selected  in  the 
study  sample  because  they  were  funded  at  less  than  $10  million.  The 
laboratory  expressed  concern  that  the  subprograms  we  sampled  produced 
only  7  percent  of  its  commercial  product-related  outputs  while 
representing  73  percent  of  its  overall  budget.  Oak  Ridge  based  this 
position  on  summary  output  data  for  the  entire  laboratory  and  sampled 
subprograms  that  laboratory  representatives  had  tabulated.  Again,  we 
could  not  include  the  output  data  for  Oak  Ridge’s  unsampled  programs  in 
our  analyses  without  violating  the  sampling  methodology.  We  also  had 
some  questions  about  the  large  number  of  outputs  the  Oak  Ridge  analysis 
ascribed  to  unsampled  programs. 

Oak  Ridge  also  thought  that  our  definitions  for  these  outputs  equated  the 
laboratories’  development  work  with  commercial  product  development. 
We  disagree.  The  definitions  we  used  make  it  clear  that  the  laboratories 
were  not  expected  to  produce  commercial  products.  Our  conclusion 
reiterates  that  the  laboratories’  outputs  related  to  commercial  product 
development  are  “precursors  to  marketable  goods”  and  that  “whether  they 
will  achieve  commercial  application  is  not  known.” 

We  provide  a  more  detailed  discussion  of  all  these  comments  and  our 
response  in  appendixes  VII  through  XVII. 

As  agreed  with  your  offices,  we  plan  no  further  distribution  of  this  report 
until  30  days  from  its  date  of  issue,  unless  you  publicly  announce  its 
contents  earlier.  We  will  then  send  copies  to  interested  parties,  and  we 
will  also  make  copies  available  to  others  upon  request. 
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If  you  have  any  questions  or  would  like  additional  information,  please  call 
me  at  (202)  512-3092.  Other  major  contributors  to  this  report  are  listed  in 
appendix  XVffl. 


Kwai-Cheung  Chan 
Director  of  Program  Evaluation  in 
Physical  Systems  Areas 
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Description  of  the  National  Laboratories 


The  descriptions  of  the  national  laboratories  are  adapted  from  the  5-year 
institutional  plans  that  the  laboratories  update  and  issue  annually  and 
from  U.S.  Department  of  Energy,  Multiprogram  Laboratories,  1979  to  1988, 
A  Decade  of  Change  (Washington,  D.C.:  Apr.  1990). 


Argonne  National 
Laboratory 


Argonne  was  established  in  1946.  The  University  of  Chicago  operates  the 
laboratory,  which  develops  and  operates  national  facilities  for  use  by 
university,  industry,  and  national  laboratory  groups;  performs  basic 
research,  technology-directed  research  and  technology  evaluations;  and 
conducts  technology  transfer  through  cooperative  research,  and 
development  agreements,  sponsored  research,  staff  exchanges,  and 
licensing  of  intellectual  property  or  through  the  formation  of  new  firms  by 
the  laboratory’s  Arch  Development  Corporation. 

The  laboratory’s  basic  research  effort  includes  experimental  and 
theoretical  research  on  fundamental  problems  in  the  physical,  life,  and 
environmental  sciences  to  advance  scientific  understanding  and  support 
energy  technology  development.  Argonne ’s  technology-directed  research 
includes  conceptualization,  design,  and  testing  of  advanced  fission 
reactors  and  other  technologies  for  power  applications  in  both  the  civilian 
and  defense  sectors  and  investigations  of  strategies  for  overcoming 
materials,  chemical,  and  electrochemical  barriers  to  the  development  of 
these  technologies.  Argonne  also  supports  doe  and,  where  appropriate, 
other  federal  agencies  in  characterizing  and  evaluating  nationally 
important  projects  and  technology  options  in  terms  of  their  environmental 
cost  or  other  implications. 


Lawrence  Berkeley 
Laboratory 


Lawrence  Berkeley,  founded  in  1931  as  the  Radiation  Laboratory  by  Ernest 
Orlando  Lawrence  of  the  University  of  California  at  Berkeley,  was  one  of 
the  original  national  laboratories.  It  was  funded  under  government 
contract  in  1942.  The  University  of  California,  which  operates  the 
laboratory,  renamed  it  the  Lawrence  Radiation  Laboratory  after  his  death 
in  1958,  and  later  called  it  Lawrence  Berkeley.  The  laboratory  conducts  a 
wide  range  of  interdisciplinary  research  with  core  competencies  in 
biosciences  and  biotechnology;  particle  and  photon  beams;  advanced 
detector  systems;  characterization  and  synthesis  of  materials;  chemical 
dynamics,  catalysis,  and  surface  sciences;  advanced  techniques  for  energy 
supply  and  energy  efficiency;  and  environmental  assessment  and 
remediation.  It  performs  research  in  the  energy,  physical,  and  life  sciences; 
develops  and  operates  national  experimental  facilities;  fosters  industry’s 
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interactions  with  the  laboratory’s  research  programs;  and  offers  scientific 
and  engineering  education  programs. 

The  laboratory’s  work  in  the  energy  sciences  includes  applied  science, 
such  as  the  energy  efficiency  of  buildings;  chemical  sciences,  such  as  the 
structure  and  reactivity  of  transient  species;  earth  sciences,  including 
geophysical  imaging  methods,  isotopic  geochemistry  and  physicochemical 
process  investigation;  and  materials  sciences,  such  as  advanced  ceramic, 
metallic,  and  polymeric  materials  for  electronic,  magnetic,  catalytic,  and 
structural  applications.  Accelerator  and  fusion  research,  nuclear  science, 
and  physics  are  pursued  in  the  general  science  area  Lawrence  Berkeley’s 
work  in  the  life  sciences  includes  cellular  and  molecular  biology,  chemical 
biodynamics,  and  research  medicine  and  radiation  biophysics.  This  work 
is  supported  by  the  laboratory’s  scientific  and  technical  resources  in  the 
areas  of  engineering,  information  and  computing  sciences,  and 
occupational  health. 


Oak  Ridge  National 
Laboratory 


Oak  Ridge  was  one  of  the  original  national  laboratories.  Now  operated  by 
Martin  Marietta  Energy  Systems,  Oak  Ridge  was  established  in  1943.  The 
laboratory’s  r&d  activities  are  focused  on  basic  and  applied  research, 
technology  development,  and  other  technological  challenges  in  areas  that 
include  energy  production  and  conservation  technologies;  experimental 
and  theoretical  research  in  physical,  chemical,  materials,  computational, 
biomedical,  earth,  environmental,  and  social  sciences;  the  design,  building, 
and  operation  of  unique  research  facilities  for  the  benefit  of  university, 
industrial,  and  other  federal  agency  and  national  laboratory  researchers; 
and  the  development  of  environmental  protection  and  waste  management 
technologies.  Oak  Ridge  also  performs  technology  transfer  and  offers 
educational  services  from  the  preschool  through  the  postdoctoral  level. 


Pacific  Northwest 
Laboratory 


Pacific  Northwest  was  established  in  1965.  Battelle  Memorial  Institute  now 
operates  the  laboratory,  which  performs  scientific  research  and  rapid 
technology  development  and  deployment  to  meet  national  needs. 
Laboratory  efforts  include  molecular  science,  hazardous  waste 
characterization,  global  environmental  studies,  subsurface  science, 
biological  systems,  technical  support  for  environmental  policies  and 
procedures,  federal  infrastructure  modernization,  national  security 
technology,  energy-efficient  methods,  advanced  analytical  methods, 
materials  research,  magnetic  fusion  research,  civilian  nuclear  waste 
management,  technical  support  for  nuclear  power  plant  operation,  space 
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exploration  technology,  fossil  fuel  technology,  renewable  energy  sources, 
energy  policy  analysis,  and  surveillance  and  oversight  of  operations  at  its 
Hanford  site. 


Idaho  National 

Engineering 

Laboratory 


This  laboratory  was  established  in  1949.  Three  contractors  operated  the 
laboratory  during  the  time  period  of  our  study:  Westinghouse  Idaho 
Nuclear  Co.,  Rockwell-INEL,  and  EG&G  Idaho.  The  laboratory’s  areas  of 
primary  emphasis  are  nuclear  reactor  technology  r&d,  defense 
production-related  support,  waste  management  and  environmental 
restoration  analysis,  advanced  energy  production  technology 
development,  and  research  and  development  on  energy  and  environmental 
issues,  including  performance  testing  of  industry-developed  electric 
vehicles,  small  hydropower  and  geothermal  power  production,  and  fossil 
energy  research.  Idaho  also  offers  educational  activities  and  performs 
technology  transfer. 


Lawrence  Livermore 
National  Laboratory 


Lawrence  Livermore  was  established  in  1952.  The  University  of  California 
operates  the  laboratory,  which  serves  as  a  national  resource  in  science 
and  engineering,  focused  on  national  security,  energy,  environment, 
biomedicine,  economic  competitiveness,  and  science  and  mathematics 
education,  with  a  special  responsibility  for  nuclear  weapons.  National 
security  has  traditionally  been  a  special  focus  of  the  laboratory’s  research 
and  development  effort.  Lawrence  Livermore’s  m^jor  areas  of  activity  have 
included  research,  development,  and  testing  for  all  phases  of  the  nuclear 
weapons  life  cycle;  strategic  defense  research;  arms  control  and  treaty 
verification  technology;  inertial  confinement  fusion;  atomic  vapor  laser 
isotope  separation;  magnetic  fusion;  other  energy  research;  research  in 
biological,  ecological,  atmospheric,  and  geophysical  sciences; 
charged-particle  beam  and  free-electron  laser  research;  advanced  laser 
and  optical  technology  applications;  technology  transfer;  and  science 
education.  The  laboratory  also  participates  in  human  genome  research  as 
part  of  a  nationally  directed  initiative. 


Los  Alamos  National 
Laboratory 


Los  Alamos,  one  of  the  original  national  laboratories,  was  established  in 
1943  and  is  operated  by  the  University  of  California.  Ensuring  the  nation’s 
deterrence  capability  through  nuclear  weapons  technology  is  the 
laboratory’s  primary  focus.  Los  Alamos’  m^jor  r&d  activities  include 
research,  design,  development,  engineering,  and  testing  of  nuclear 
warheads;  maintenance  and  enhancement  of  the  weapons  technology  base 
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and  warhead  stockpile  management;  research,  development,  and  testing 
support  for  advanced  nuclear  directed-energy  concepts;  nuclear  materials 
r&d  for  the  nuclear  weapons  program;  nonnuclear  strategic  defense  r&d 
activities;  advanced  conventional  munitions  development  and  simulation; 
verification  and  safeguards  r&d;  vulnerability,  lethality,  effects,  and 
countermeasures  research;  advanced  defense  technologies;  intelligence 
activities  involving  hardware  analysis  and  technology  security;  weapons 
and  energy  technology  systems  studies;  and  r&d  in  nonnuclear  energy  and 
technology  areas.  The  laboratory’s  basic  research  activities  in  defense  and 
energy  areas  include  atomic  and  molecular  physics,  bioscience,  chemistry, 
computational  science  and  applied  mathematics,  geoscience,  space 
science,  astrophysics,  materials  science,  nuclear  and  particle  physics, 
plasma  physics,  fluids,  and  particle  beams.  Los  Alamos  also  performs 
technology  transfer  and  offers  science  and  engineering  education 
programs. 


Sandia  National 
Laboratories 


> 


Sandia  was  established  in  1949  under  an  agreement  with  AT&T  to  operate 
the  laboratory  for  the  government  as  a  public  service  on  a  nonprofit  basis. 
at&t  stepped  out  of  this  role  in  1993.  A  contract  was  recently  awarded  to 
Martin  Marietta  Corporation  to  operate  the  laboratories.  Sandia’s  major 
areas  of  effort  are  nuclear  weapons,  arms  control  and  treaty  verification, 
environmental  restoration  and  waste  management,  energy  supply  and 
conservation,  advanced  conventional  military  technologies,  and  other 
programs  in  the  national  interest.  The  laboratories’  r&d  activities  in  these 
areas  include  research,  development,  and  engineering  associated  with 
advancing  nuclear  explosives  to  integrated,  functional  weapons  for 
Department  of  Defense  weapon  delivery  systems;  other  defense  programs, 
including  development  of  verification  and  control  technologies  to  support 
arms  reduction  and  concepts  and  systems  for  the  safeguarding  and 
security  of  nuclear  materials;  research,  development,  and  engineering  for 
hazardous  waste  removal,  minimization,  and  remediation;  and  nonnuclear 
energy  research  in  energy  efficiency,  recovery  techniques,  conversion 
technologies,  alternative  energy  sources,  characterization  of 
environmental  change  phenomena,  environmental  restoration 
technologies,  and  basic  energy  sciences.  Sandia  also  conducts  technology 
transfer  and  offers  mathematics  and  science  education  opportunities. 


Brookhaven  National 
Laboratory 


Brookhaven  was  established  in  1947  by  a  group  of  nine  universities  to 
facilitate  their  mutual  access  to  large-scale  research  facilities,  particularly 
in  nuclear  science.  The  laboratory  is  operated  by  Associated  Universities, 
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a  corporation  governed  by  a  board  of  trustees  representing  the  original 
nine  universities  as  well  as  other  universities,  research  institutions,  and 
industrial  organizations.  Brookhaven’s  primary  role  is  to  conceive,  design, 
build,  and  operate  large-scale,  complex  facilities  for  scientific  research 
and  to  conduct  basic  and  applied  research  in  energy-related  physical,  life, 
and  environmental  sciences.  When  feasible,  Brookhaven  makes  its 
laboratory  facilities  available  to  state  and  federal  agencies,  universities, 
and  private  industry.  The  laboratory’s  major  areas  of  r&d  are  high-energy 
and  nuclear  physics;  basic  energy  sciences  emphasizing  research  on 
biological,  chemical,  and  physical  phenomena  underlying  energy-related 
transfer,  conversion,  and  storage  systems;  life  sciences,  nuclear  medicine, 
and  medical  applications  of  nuclear  techniques;  and  a  broad  span  of 
applied  programs  that  draw  on  the  laboratory’s  unique  capabilities. 
Brookhaven  makes  all  useful  results  and  knowledge  obtained  from  its 
research  activities  available  to  private  industry.  Brookhaven  also  performs 
technology  transfer  and  offers  science  and  engineering  education 
programs. 


National  Renewable 
Energy  Laboratory 


The  former  Solar  Energy  Research  Institute  was  designated  a  doe  national 
laboratory  in  1991  and  renamed  the  National  Renewable  Energy 
Laboratory.  The  focus  of  the  laboratory’s  effort  is  on  developing 
competitive  renewable  energy  and  related  technologies  and  facilitating 
their  commercialization.  The  laboratory’s  r&d  activities  include  basic  and 
applied  research,  exploratory  and  advanced  development  and  other 
activities  in  renewable  energy  and  related  technologies;  analytic  studies 
and  technology  evaluations;  and  collaborative  R&D  with  universities  and 
industry.  The  laboratory  also  manages  subcontracted  R&D  on  behalf  of  doe 
and  serves  as  a  source  of  scientific  and  technical  information  on 
renewable  energy. 
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Unitid  States  General  Accounting  Office 


RATIONAL  LABORATORY  INVENTORY 


PA«T  1:  RESEARCN  PROGRAMS 


mm  OF  TIC  MKT 

The  Congress  hee  asked  the  U.*.  General  Accounting  Office  to  develop  an  inventory  of  the  funding  and 
personnel  for  the  10  U.S.  Department  of  Energy  national  laboratories'  research  program  as  net  l  as  the 
facilities  and  equipsient  that  support  theai.  The  purpose  of  this  survey  is  to  collect  detailed  information 
for  the  laboratory  Inventory  aa  wall  aa  preliminary  information  about  tachnology  tranafar  activities.  Wa 
uill  conduct  fol  lou-up  intervieua  uith  each  laboratory's  raprssantatives  in  order  to  establish  the 
laboratory's  perspective  on  its  research  mission  end  the  potential  for  cooperative  efforts  uith  industry. 


aou  to  mrmE  trie  guest iormaire 


The  questionnaire  is  divided  into  tuo  parts-part  I  concerns  the  laboratory's  rasaareh  programs;  part  II 
requests  infonsstlon  about  the  laboratory's  major  research  facilities.  Farts  1  and  II  are  being  sent  to 
laboratory  officials  designated  by  laboratory  msnagssmnt  as  knouledgeabls  in  those  areas.  This  is  part  1 
and  should  be  cosplated  by  research  program  directors  or  the  sqivalent.  It  contains  questions  about  the 
laboratory's  research  progress.  Rany  of  the  questions,  psrticularly  those  concerning  shipment, 
expenditures,  and  personnel,  uill  require  a  search  of  the  laboratory's  records  to  complete.  It  may  help  to 
read  the  questionnaire  through  quickly  first  so  that  you  uill  knou  uhat  information  you  may  have  to  look  up 
before  you  start  to  anauer  individual  questions. 


If  you  have  any  quest  Iona  uhile  completing  your  responses,  plaass  contact  Nancy  iriggs,  on  (202)  275-170J  or 
Sara  Edoondson,  on  (202)  275-8516  in  the  Program  Evaluation  and  Methodology  Division,  U.S.  General 
Accounting  Office. 


Please  provide  the  name,  title,  telephone  nurtxr  and  FAX  nueber  of  the  person  who  may  be  contacted 
concerning  the  responses  to  this  cfjestiomafre: 


Contact  person's  name: 


Title: 


Telephone  Xtmtotr:. 
FAX  Nuiber:  _ 
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Please  return  the  completed  questionnaire  in  the  enclosed  envelope  within  10  working  days  of  receipt.  In 
the  event  that  the  enclosed  envelope  is  misplaced,  please  mail  the  questionnaire  to: 

Nancy  Briggs,  Ph.D 
Project  Manager 

U.5.  General  Accounting  Office 

Program  Evaluation  and  Methodology  Division 

Room  5853 

441  G  Street  St. ,  N.W. 

Washington,  DC  20548 

Thank  you  in  advance  for  your  cooperation  and  assistance  in  addressing  an  issue  of  such  critical  importance 
to  the  nation.  We  will  send  a  report  on  the  analysis  of  the  information  to  the  Congress  and  you. 
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DEFINITIONS 

The  definitions  listed  below  are  included  to  provide  a  common  frame  of  reference  for  responding  to  the 
survey . 

Applied  Research:  A  study  directed  toward  the  practical  use  of  knowledge  or  understanding  of  a  subject  to 
meet  a  recognized  need. 

Sasic  Research:  A  study  undertaken  primarily  to  gain  fuller  knowledge  or  understanding  of  a  subject  and  to 
contribute  to the  knowledge  base  in  the  field  of  investigation. 

capital  Equipment  Budget:  For  a  national  laboratory,  this  includes  the  budget  for  both  research  equipment 
and  general  purpose  equipment . 

Cooperative  Research  and  Development  Agreement  (CRDA) :  A  contractual  form  created  for  the  express  purpose 
of  fostering  technology  transfer  from  the  federal  domain  to  the  private  sector . 

Development:  Research  directed  toward  the  production  of  useful  materials,  devices,  systems,  or  methods, 
including  the  design  and  development  of  prototypes  and  processes. 

Facility:  An  entity  that  comprises  the  equipment  used  for  research  programs.  A  facility  may  include  a 
building  or  defined  structure,  some  area  within  a  structure,  or  a  defined  area  not  confined  to  a  structure 
(for  example,  a  testing  area) . 

Laboratory:  A  group  of  facilities  owned,  leased,  or  otherwise  used  by  the  U.S.  Department  of  Energy  to 
perform  research  and  development.  A  laboratory  consists  of  land  (including,  but  not  limited  to,  remote 
testing  areas),  buildings,  human  resources,  research  programs,  and  equipment. 

Mission:  The  primary  scientific  and  technical  research  programs  that  a  laboratory  pursues. 

Operating  Budget:  For  a  national  laboratory  this  comprises  research  and  development  program  costs, 
including  salaries  and  wages,  expendables  and  overhead. 

Research  and  Development:  Intensive,  systematic  study  directed  toward  fuller  scientific  knowledge  or 
understanding  of  the  subject  under  investigation;  practical  use  of  knowledge;  or  the  production  of 
materials,  devices,  systems,  or  methods.  Research  and  development  includes  basic  research,  applied 
research,  and  development. 

Research  Program:  One  of  several  broad  areas  of  research  activity  within  a  laboratory's  mission. 

Technology:  Devices,  processes,  “know  how,*  or  scientific  and  technical  information  produced  through  the 
research  and  development  process. 

Technology  Transfer:  The  process  that  fosters  the  use  of  devices,  processes,  "know  how,*  or  scientific  and 
technical  information  produced  in  a  federal  laboratory  by  universities,  private  industry,  or  government 
agencies,  whether  national  (federal,  state,  or  local)  or  foreign. 

Total  Budget:  For  a  national  laboratory,  this  includes  operating,  capital  equipment,  and  construction 
costs. 

User  Facility:  A  federal  laboratory  facility  available  for  use  either  free  of  charge  or  on  a  cost- 
reimbursed  basis  by  investigators  from  private  industry,  academic  institutions,  or  state  and  local 
government  agencies. 
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PROGRAM  AND  SUBPROGRAM  DESCRIPTION 

i,  please  identify  the  defense  subprogram  for  which  you  are  reporting.  Report  only  for  a  subprogram  funded  by  the  laboratory's 
operating  budget  during  fiscal  year  1992  and  which  is  on-going  for  fiscal  year  1993.  {Check  only  one.) 


GB-011  Weapons  RD&T 


GB-02  inertial  Confinement  Fusion  (Required) 


GC  Verification  and  Control  Total 


Other  (Please  specify) 


‘U.S.  Department  of  Energy  Budget  and  Reporting  and  System  code. 
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4.  How  important,  if  at  all,  is  each  of  the  research  and  development-related  activities  listed  below  to  your  subprogram 
objectives?  (Check  the  relevant  space.) 


Research  and  development -related 
activity 

Importance  j 

Little 

importance 

(1) 

Somewhat 

important 

(2) 

Moderately 

important 

(3) 

Very 

important 

(4) 

Essential 

(5) 

Not  part 
of  mission 
(6) 

a.  Basic  research1 

b.  Applied  research2 

c .  Development1 

d.  Providing  technical  assistance  to 
government  agencies 

e.  Providing  technical  assistance  to 
universities 

f.  Providing  technical  assistance  to 
private  firms  or  industrial 
organizations 

g.  Transfer  technology  from  this 
laboratory  to  U.S.  government 
organizations 

h.  Transfer  technology  from  this 
laboratory  to  foreign  government 
organizations 

i.  Transfer  technology  from  this 
laboratory  to  U.S.4  firms  or  industrial 
organizations 

j.  Transfer  technology  from  this 
laboratory  to  foreiqn*  firms  or 
industrial  organizations 

k.  Other  (Please  specify) 

Masic  Research:  A  study  undertaken  primarily  to  gain  fuller  knowledge  or  understanding  of  a  subject  and  to  contribute  to  the 
knowledge  base  in  the  field  of  investigation. 

^Applied  Research:  A  study  directed  toward  the  practical  use  of  knowledge  or  understanding  of  a  subject  to  meet  a  recognized  need. 

^Development :  Research  directed  toward  the  production  of  useful  materials,  devices,  systems,  or  methods,  including  the  design  and 
development  of  prototypes  and  processes. 

Majority-owned  by  U.S.  citizens 

Majority-owned  by  non-U. S.  citizens 
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5.  In  your  estimation,  what  percent,  if  any,  of  your  total  research  subprogram  budget  was  expended  each  year  during  fiscal  years 
1989-92  for  each  of  the  research  and  development-related  activities  listed  below?  {Annual  total  should  equal  100  percent.  Write 
*N/A“  if  the  program  was  not  in  operation  in  given  year.) 


Research  and  development-related 
activity 

1992 

1991 

1990 

1989 

Basic  research1 


Applied  research2 


Deve 1 opment3 


d.  Providing  technical 
assistance  to  government 
agencies 


e.  Providing  technical 
assistance  to  universities 


f .  Providing  technical 
assistance  to  private  firms  or 
industrial  organizations 


i.  Transfer  technology  from 
this  laboratory  to  U.S.*  firms  or 
industrial  organizations  _ 


iflasic  Research:  A  study  undertaken  primarily  to  gain  fuller  knowledge  or  understanding  of  a  subject  and  to  contribute  to  the 
knowledge  base  in  the  field  of  investigation. 

2 Applied  Research:  A  study  directed  toward  the  practical  use  of  knowledge  or  understanding  of  a  subject  to  meet  a  recognized  need. 

development:  Research  directed  toward  the  production  of  useful  materials,  devices,  systems,  or  methods,  including  the  design  and 
development  of  prototypes  and  processes. 

4Majority-owned  by  U.S.  citizens 
'Majority-owned  by  non-U. S.  citizens 
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6.  We  identified  the  technologies  listed  below  using  Che  Report  of  the  National  critical  Technologies  Panel.  During  fiscal  years 
1989-92,  which  of  these  technologies  did  the  research  of  this  subprogram  support? In  your  estimation  what  percentage  of  the 
research  subprograms  budget  was  spent  each  year  for  research  about  these  technologies?  (Check  all  that  apply.  Write  in  the 
estimated  percentage  for  activities  checked.) 


1992 


a. 

□ 

Materials  synthesis  and 
processing 

b. 

□ 

Electronic  and  photonic 
materials 

c. 

□ 

Ceramics 

d. 

□ 

Composites 

e. 

□ 

High-performance  metals  and 
alloys 

f . 

□ 

Flexible  computer 
integrated  manufacturing 

9- 

□ 

Intelligent  processing 
equipment 

h. 

□ 

Micro  -  and  nanofabrication 

i. 

□ 

System  management 
technologies 

j. 

□ 

Software 

k. 

□ 

Microelectronics  and 
optoelectronics 

1. 

□ 

High-performance  computing 
and  networking 

m. 

□ 

High  definition  imaging 
and  displays 

n. 

□ 

Sensors  and  signal 
processing 

0. 

□ 

Data  storage  and 
peripherals 

P- 

□ 

Computer  simulation  and 
modeling 

1-9 
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q.  |— j  Applied  molecular  biology 

r.  j— j  Medical  technology 

s.  i — |  Aeronautics 


t.  | — I  Surface  transportation 
' — 1  technologies 


u.  j— j  Energy  technologies 


v.  j— i  Pollution  minimization, 
* — 1  remediation,  and  waste 

management 

w.  j  Other  (Please  specify) 
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7.  How  many  of  the  research  and  development-related  outputs  listed  below  did  your  subprogram  produce  each  year  from  fiscal  year 
1989  to  1992?  Write  ‘N/A’  if  the  program  was  not  in  operation  in  a  given  year.) 

1992  1991  1990  1989 

a.  Published  books  _  _  _  _ 

b.  Published  articles  . .  . .  .  . 

c.  Published  book  chapters  _  ,  -  - 

d.  Technical  and  scientific 
reports/monographs  for  internal 
use  only 

e.  Technical  and  scientific 
reports /monographs  for  release 
to  others  outside  the 

laboratory  -  -  -  - 

f.  Prototype  devices  and  materials  -  -  -  - 

g.  Papers  for  presentation  at 

professional  conferences  -  -  -  - 

h.  New  program  - - -  - -  -  - 

i.  Algorithms  -  - - - - -  -  - - 

j .  Software  -  -  ■ — -  — - 

k.  Invention  disclosures  _  _  _ — -  - - 

l.  Patents  -  -  -  - 

m.  Licenses  -  -  -  - 

n.  Commercial  products  _  _  _  _ 

o.  Commercial  processes  -  -  -  - 

p.  Spin-off  company  -  -  -  - 

q.  Other  (Please  specify)  _  _  _  _ 
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8.  How  much  royalty  income  was  earned  each  year  during  fiscal  years  1989-92  by  technologies  supported  by  this  subprogram?  Show 
dollars  in  millions.  (Write  *N/A"  if  the  program  was  not  in  operation  in  a  given  year.) 


1992  1991  1990  1989 


Royalty  income  earned  $. 


9.  What  was  the  total  number  of  CRDAs  and  -Work  for  Others*  contracts  in  effect  for  this  subprogram  each  year  during  fiscal  years 
1989-92? 


1992  1991  1990  1989 


a.  CRDAs 

Work  for  others 


b.  Federal  agency  other 
than  DOE 


c.  Industry 


d.  Foreign  governments 


e.  Other  sources 
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10.  During  what  time  period,  if  at  all,  does  this  research  subprogram  have  the  potential  for  industrial  application 
product  development  (Check  only  one) . 

a.  r— |  Immediate  future  (Less  than  5  years) 

b.  |— j  Short-term  future  (Next  5-10  years) 

c.  |— |  Long-term  future  (Over  10-20  years) 

d.  [— |  Very  long-term  future  (Over  20  years) 


commercial 
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11.  is  any  university  or  private  industry  working  in  cooperation  with  your  research  subprogram? 

a .  j  Yes 

b.  | — j  No 

If  yes,  please  provide  the  following  information: 

c.  Organization  name:  ... _ _ _ 

d.  Address:  _ „ _ _ _ _ — — 

Contact  person:  _ _ _ 

f.  Telephone  number:  _ _ _ 


g.  Research  topic  of  cooperative  effort:  . . . — - - 

Please  also  indicate  if  this  is  a  U.S.  or  foreign  government,  university  or  firm.  (Check  the  relevant  space.) 


Country 

Organization 

Government  Agency 
(1) 

University 

(2) 

Firm  or  Company 
(3) 

h.  U.S. 

If  there  is  more  than  one  organization  working  in  cooperation  with  your  subprogram,  please  photocopy  this  page  and  provide  this 
information  for  all  organizations. 
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12.  If  possible,  please  identify  up  to  three  other  U.S.  or  foreign  government ,  university,  or  industrial  research  institutions 
that  in  your  view  are  working  on  scientific  and  technical  problems  similar  to  those  addressed  by  this  research  subprogram. 


similar  to  those  addressed  by  this  research  subprogram. 


Name  of  scientific  and  technical 
problem 


Name  of  other  government, 
university  or  industrial  R6D 
institution 


Contact  person's  name  and 
telephone  number 
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13  if  possible  please  identify  one  or  two  administrators  of  research  programs  in  U-S-  government,  university,  or  industrial 
research^institutions  who  are  fLiliar  with  the  scientific  and  technical  aspects  of  your  research  subprogram  as  «  t*« 

facilities  and  equipment  that  support  it.  please  do  not  list  anyone  who  is  affiliated  with  programs  in  the  U.S.  Department  or 
Energy. 


a.  Research  administrator's  name  and  title 


Name :  _ _ _ _ _ _ 

Title:  _ _ _ ■ 

Organization  name,  address  and  phone  number 
Name:  _ _ _ — - 

Address: _ _ _ - _ 

Telephone  no:  _ _ 

FAX  no:  _ _ _ _ — 


b.  Research  administrator's  name  and  title 

Name:  _ _ _ _ _ _ 

Title: _ _ _ _ _ 

Organization  name,  address,  and  phone  number 
Name:  _ _ _ _ _ _ 

Address:  - - - 

Telephone  no: _ _ _ 

FAX  no:  _ _ _ _ _ 
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PROGRAM  BUDGET 

{Dollars  in  millions} 

1992  1991  1990  1989 

14.  What  was  your  research  subprogram's  total  annual 
budget  during  fiscal  years  19B9-92?  Show  dollars  in 
millions.  (Write  the  dollar  amount  in  each  column. 

Write  ■N/A"  if  the  subprogram  was  not  in  operation  in 
a  given  year.) 


Operating 

a.  Salaries  and  wages 

b.  All  other 

c .  Capital  equipment 

d.  Construction 

e.  Other  (Please  specify) 


15.  What  percentage  of  the  research  subprogram's  total  annual  budget  was  contributed  by  each  of  the  following  sources  during 
fiscal  years  1989-92?  (Write  'N/A*  if  the  subprogram  was  not  in  operation  in  a  given  year.  Annual  total  should  equal  100 
percent. ) 


1992  1991  1990  1989 


a.  Contracts  from  DOE 

Work  for  Others 

b.  Federal  agencies  other  than  DOE 

c.  U.S.  industries 

d.  Foreign  industry 


1-17 
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PERSONNEL 

16.  How  many  workers  are  employed  annually  by  your  research  subprogram  in  each  of  the  following  job  categories  and  what  is  the 
total  employment?  Please  provide  both  the  number  of  full-time  personnel  and  the  number  of  full-time  equivalent  (FTE)  staff  years 
for  each  year  during  fiscal  years  1989-92. 


Job  category 

1992 

1991 

1990 

1989 

Number  of 
personnel 

FTEs 

Number  of 
personnel 

FTEs 

Number  of 
personnel 

FTEs 

Number  of 
personnel 

FTEs 

a.  Administrators  (not 
involved  in  research) 

b.  Scientists,  engineers, 
and  other  researchers 
(including  research 
administrators  directly 
involved  in  research) 

c.  Technicians  supporting 
research  (through  testing, 
inspection,  maintenance,  or 
construction  of  research 
equipment,  computer 
programming) 

d.  Clerical  maintenance  and 
other  support  personnel 

e.  Other  (Please  specify) 

f.  Total  employees 

THANK  YOU  FOR  YOUR  PARTICIPATION 
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Survey  Response 

The  research  program  survey  population  was  enumerated  by  applying 
selection  criteria  to  each  laboratory’s  research  programs.  After  processing 
the  surveys,  we  sent  the  laboratories  a  letter  requesting  confirmation  that 
our  list  of  research  programs  and  subprograms  was  complete.  In  response 
to  our  letter,  the  laboratories  confirmed  a  total  of  252  research 
subprograms.  The  laboratories  returned  a  total  of  247  data  collection 
instruments,  for  a  survey  response  rate  of  98  percent. 

Data  Quality  Issues 

■ -  "  "  ”  ~  | 

The  data  contained  in  this  report  are  results  of  analyses  of  national 

laboratory  program  managers’  responses  to  questions  5,  6,  7, 9,  and  10  in 
part  I  of  the  national  laboratory  inventory.  These  responses  represent 
program  managers’ judgments  or  self-reports  about  question  elements,  as 
follows.  We  made  no  attempt  to  validate  these  responses  through 
independent  sources. 

Question  5 

Responses  are  research  program  managers’  best  estimates  of  the 
proportion  of  the  total  program  budget  expended  for  each  R&D-related 
activity.  Although  they  had  our  definitions  for  key  R&D-related  activities 
listed  in  the  data  collection  instrument,  their  responses  also  may  reflect 
their  own  understanding  of  terms  such  as  basic  research,  applied  research, 
or  technical  assistance. 

Question  6 

Responses  are  research  program  managers’  best  estimates  of  the 
proportion  of  the  total  program  budget  expended  for  research  in  critical 
technology  areas.  The  response  categories  in  the  question  are  the  critical 
technologies  identified  by  the  National  Critical  Technologies  Panel.  Some 
overlap  may  exist  among  these  categories  because  they  were  not  identified 
for  research  measurement  puiposes.  The  Panel’s  critical  technology 
categories  were  used  in  this  question  to  determine  the  congruence 
between  research  already  being  conducted  at  the  national  laboratories  and 
the  research  needs  articulated  by  a  congressionally  mandated  body. 

A  few  responses  submitted  for  this  question  summed  to  more  than 

100  percent.  These  responses  were  prorated  to  include  them  in  the 
calculation  of  mean  percent  expenditures  for  r&d  in  critical  technologies. 

Question  7 

Responses  are  research  program  managers’  reports  of  research  program 
outputs.  The  responses  concerning  commercial  products  and  commercial 
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processes  are  judgments  made  about  research  outputs  that  have  reached 
only  the  precompetitive  stage  of  the  r&d  process. 

Question  9 

Responses  are  research  program  managers’  reports  about  cradas  in  effect 
through  the  end  of  fiscal  year  1992. 

Question  10 

Responses  are  research  program  managers’  judgments  about  potential 
industrial  application  or  commercial  product  development  for  outputs  of 
their  research  program  over  a  20-year  planning  horizon.  The  size  of 
research  subprograms  in  the  study  population  varied;  thus,  managers  were 
considering  outputs  of  one  or  more  research  activities  in  making  their 
assessments. 
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R&D  Activities 


Data  concerning  the  distribution  of  the  national  laboratories’  expenditures 
among  R&D-related  activities,  by  laboratory  and  by  program,  are  presented 
below. 


Table  IV.1:  Mean  Percent  Expenditures  for  R&D-Related  Activities  Within  Laboratories8 

R&D-related  activity 

ANL 

LBL 

ORNL 

PNL 

INEL 

LLNL 

LANL 

SNL 

BNL 

NREL 

All  labs 

Basic  research 

29.9 

39.7 

25.1 

9.8 

2.2 

7.6 

19.4 

4.9 

53.6 

6.9 

17.4 

Applied  research 

22.5 

15.9 

29.1 

29.7 

20.6 

36.2 

33.1 

23.5 

7.5 

24.3 

27.2 

Development 

17.4 

7.4 

17.9 

31.8 

25.5 

43.6 

25.9 

47.7 

13.9 

31.9 

30.9 

Technical  assistance  to 

Government  agencies 

10.8 

4.3 

5.9 

21.9 

15.8 

6.2 

8.6 

11.9 

7.8 

4.5 

9.7 

Universities 

3.5 

11.0 

6.2 

1.0 

1.6 

0.6 

1.4 

0.9 

5.7 

3.3 

2.5 

Private  firms  or  industrial 
organizations 

1.6 

3.5 

5.0 

1.2 

1.4 

0.8 

1.7 

2.0 

2.9 

7.9 

2.2 

Technology  transfer  to 

Government  agencies 

4.0 

10.4 

4.5 

2.4 

7.0 

1.9 

2.5 

3.1 

1.6 

1.7 

3.2 

Private  firms  or  industrial 
organizations 

6.2 

1.7 

6.1 

2.1 

4.6 

2.1 

3.3 

3.9 

0.9 

18.1 

3.8 

Otherb 

4.1 

6.1 

0.2 

0 

21.3 

1.0 

4.1 

2.1 

6.2 

1.5 

3.0 

Total 

100 

100 

100 

100 

100 

100 

100 

100 

100 

100 

Legend 


ANL  =  Argonne  National  Laboratory 

LBL  =  Lawrence  Berkeley  Laboratory 

ORNL  =  Oak  Ridge  National  Laboratory 

PNL  =  Pacific  Northwest  Laboratory 

INEL  =  Idaho  National  Engineering  Laboratory 

LLNL  =  Lawrence  Livermore  National  Laboratory 

LANL  =  Los  Alamos  National  Laboratory 

SNL  =  Sandia  National  Laboratories 

BNL  =  Brookhaven  National  Laboratory 

NREL  =  National  Renewable  Energy  Laboratory 

aFiscal  year  1992. 

bSubprogram  expenditures  for  activities  other  than  those  listed  above,  such  as  training  graduate 
students  and  postdoctoral  fellows  or  safety  procedures. 
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Table  IV.2:  Mean  Percent  Expenditures  for  R&D-Retated  Activities  Among  Laboratories8 


R&D-related  activity 

ANL 

LBL 

ORNL 

PNL 

INEL 

LLNL 

LANL 

SNL 

BNL 

NREL 

Total 

Basic  research 

14.9 

10.9 

13.1 

5.0 

0.3 

8.3 

20.7 

5.3 

20.4 

1.2 

100 

Applied  research 

7.2 

2.8 

9.7 

9.6 

2.0 

25.3 

22.6 

16.3 

1.8 

2.7 

100 

Development 

5.0 

1.1 

5.2 

9.1 

2.2 

26.8 

15.6 

29.1 

3.0 

3.1 

100 

Technical  assistance  to 

Government  agencies 

9.7 

2.1 

5.6 

20.0 

4.2 

12.1 

16.5 

23.2 

5.3 

1.4 

100 

Universities 

12.1 

20.9 

22.2 

3.3 

1.6 

4.3 

10.2 

6.9 

14.8 

3.9 

100 

Private  firms  or  industrial 
organizations 

6.4 

7.6 

20.8 

4.8 

1.7 

7.3 

14.7 

17.1 

9.0 

10.7 

100 

Technology  transfer  to 

Government  agencies 

10.8 

15.4 

12.6 

6.7 

5.6 

11.4 

14.5 

18.2 

3.3 

1.6 

100 

Private  firms  or  industrial 
organizations 

14.2 

2.2 

14.5 

4.8 

3.1 

10.4 

16.0 

19.3 

1.6 

14.0 

100 

Otherb 

11.9 

9.6 

0.7 

0.0 

18.1 

6.4 

25.2 

13.1 

13.6 

1.5 

100 

All  activity 

8.7 

4.8 

9.1 

8.8 

2.6 

19.0 

18.6 

18.9 

6.6 

3.0 

100 

Legend 


ANL  =  Argonne  National  Laboratory 

LBL  =  Lawrence  Berkeley  Laboratory 

ORNL  =  Oak  Ridge  National  Laboratory 

PNL  =  Pacific  Northwest  Laboratory 

INEL  =  Idaho  National  Engineering  Laboratory 

LLNL  =  Lawrence  Livermore  National  Laboratory 

LANL  =  Los  Alamos  National  Laboratory 

SNL  =  Sandia  National  Laboratories 

BNL  =  Brookhaven  National  Laboratory 

NREL  =  National  Renewable  Energy  Laboratory 

aFiscal  year  1992. 

bSubprogram  expenditures  for  activities  other  than  those  listed  above,  such  as  training  graduate 
students  and  postdoctoral  fellows  or  safety  procedures. 
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Table  IV.3:  Mean  Percent  Expenditures  for  R&D-Related  Activities  Within  Programs8 


R&D-related  activity 

ER 

CE 

ES&H 

NE 

DP 

NPR 

ERWM 

FE 

CRWM 

PPA 

INT 

WFO 

All 

programs 

Basic  research 

53.1 

4.4 

0 

7.6 

7.2 

0 

0.3 

14.1 

8.2 

3.3 

2.0 

11.5 

17.4 

Applied  research 

15.2 

25.9 

8.8 

29.4 

32.5 

7.6 

32.4 

31.5 

52.8 

43.3 

50.0 

26.1 

27.2 

Development 

10.1 

28.6 

9.8 

51.7 

40.9 

27.7 

37.3 

15.8 

17.3 

6.7 

15.0 

23.4 

30.9 

Technical  assistance  to 

Government 

agencies 

4.0 

5.9 

71.8 

1.6 

9.7 

0 

10.1 

10.3 

11.6 

46.7 

25.0 

25.1 

9.7 

Universities 

6.5 

4.0 

2.7 

0.6 

0.7 

0 

1.2 

2.9 

0.4 

0 

0 

2.8 

2.5 

Private  firms  or 

industrial 

organizations 

2.8 

11.6 

0.1 

0.6 

0.9 

0 

0.7 

5.8 

0 

0 

0 

2.2 

2.2 

Technology  transfer  to 

Government 

agencies 

4.1 

3.1 

2.4 

2.0 

2.8 

0 

3.3 

7.7 

1.7 

0 

8.0 

4.3 

3.2 

Private  firms  or 

industrial 

organizations 

2.5 

15.7 

0.7 

6.5 

2.5 

0 

2.8 

7.1 

0 

0 

0 

4.4 

3.8 

Otherb 

1.7 

0.7 

3.7 

0 

2.8 

64.7 

11.8 

4.9 

8.0 

0 

0 

0.4 

3.0 

Total 

100 

100 

Legend 


ER  =  Energy  Research 

CE  =  Conservation  and  Renewable  Energy 

ES&H  =  Environment,  Safety  and  Health 

NE  =  Nuclear  Energy 

DP  =  Defense  Programs 

NPR  =  New  Production  Reactors 

ERWM  =  Environmental  Restoration  and  Waste  Management 
FE  =  Fossil  Energy 

CRWM  =  Civilian  Radioactive  Waste  Management 
PPA  =  Policy  Planning  and  Analysis 
I  NT  =  Intelligence 
WFO  =  Work  for  Others 

“Fiscal  year  1992. 

bSubprogram  expenditures  for  activities  other  than  those  listed  above,  such  as  training  graduate 
students  and  postdoctoral  fellows  or  safety  procedures. 
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Table  IV.4:  Mean  Percent  Expenditures  for  R&D-Related  Activities  Among  Programs® 

R&D-related  activity 

ER 

CE 

ES&H 

NE 

DP 

NPR 

ERWM 

FE 

CRWM 

PPA 

INT 

WFO 

Total 

Basic  research 

68.7 

1.7 

0 

3.0 

17.8 

0 

0.1 

0.2 

0.6 

0 

0 

7.7 

100 

Applied  research 

12.6 

6.4 

0.1 

7.3 

51.3 

0.2 

7.9 

0.3 

2.4 

0.1 

0.1 

11.2 

100 

Development 

7.3 

6.2 

0.1 

11.3 

56.7 

0.6 

8.0 

0.1 

0.7 

0 

0 

8.9 

100 

Technical  assistance  to 

Government  agencies 

9.4 

4.1 

3.0 

1.1 

43.1 

0 

6.9 

0.3 

1.5 

0.3 

0.1 

30.3 

100 

Universities 

57.9 

10.8 

0.4 

1.6 

12.5 

0 

3.2 

0.3 

0.2 

0 

0 

12.8 

100 

Private  firms  or  industrial 
organizations 

29.1 

36.3 

0 

1.9 

17.6 

0 

2.3 

0.7 

0 

0 

0 

12.0 

100 

Technology  transfer  to 

Government  agencies 

28.4 

6.4 

0.3 

4.1 

36.9 

0 

6.9 

0.6 

0.6 

0 

0.1 

15.5 

100 

Private  firms  or  industrial 
organizations 

14.5 

27.8 

0.1 

11.5 

27.5 

0 

4.8 

0.5 

0 

0 

0 

13.4 

100 

Otherb 

13.0 

1.6 

0.5 

0 

39.2 

14.6 

25.9 

0.4 

3.3 

0 

0 

1.4 

100 

All  activity 

22.5 

6.8 

0.4 

6.8 

42.9 

0.7 

6.6 

0.3 

1.2 

0.1 

0 

11.7 

100 

Legend 


ER  =  Energy  Research 

CE  =  Conservation  and  Renewable  Energy 

ES&H  =  Environment,  Safety  and  Health 

NE  =  Nuclear  Energy 

DP  =  Defense  Programs 

NPR  =  New  Production  Reactors 

ERWM  =  Environmental  Restoration  and  Waste  Management 
FE  =  Fossil  Energy 

CRWM  =  Civilian  Radioactive  Waste  Management 
PPA  =  Policy  Planning  and  Analysis 
INT  =  Intelligence 
WFO  =  Work  for  Others 

“Fiscal  year  1992. 

bSubprogram  expenditures  for  activities  other  than  those  listed  above,  such  as  training  graduate 
students  and  postdoctoral  fellows  or  safety  procedures. 


Page  53 


GAO/PEMD-95-2  National  Laboratories’  R&D  Activities 


Appendix  V _ _ _ _ _ _ _ 

National  Laboratories’  R&D  Outputs 


Data  concerning  outputs  of  the  national  laboratories’  R&D-related  activities, 
by  laboratory  and  by  program,  are  presented  below. 
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Table  V.1 :  National  Laboratories’  Research  Program  Outputs  by  Laboratory8 


Outputs 

ANL 

LBL 

ORNL 

PNL 

INEL 

LLNL 

LANL 

SNL 

BNL 

NREL 

Total 

Publications  and  reports 

Books 

8 

2 

16 

1 

3 

17 

14 

10 

14 

9 

94 

Articles 

738 

1,634 

955 

210 

79 

461 

1,134 

534 

634 

92 

6,471 

Book  chapters 

71 

35 

154 

11 

5 

51 

25 

19 

b 

b 

371 

Reports 

540 

620 

1,200 

170 

335 

1,405 

828 

1,009 

515 

180 

6,802 

Conference  papers 

662 

1,027 

1,437 

369 

245 

747 

1,619 

968 

534 

247 

7,855 

Subtotal 

2,019 

3,318 

3,762 

761 

667 

2,681 

3,620 

2,540 

1,697 

528 

21,593 

Outputs  related  to  commercial 
product  development  Prototypes 

28 

14 

27 

31 

28 

68 

75 

233 

50 

12 

566 

Algorithms 

11 

6 

6 

10 

1 

21 

52 

646 

17 

8 

778 

Software 

36 

11 

7 

3 

120 

57 

96 

193 

23 

8 

554 

Patents 

14 

12 

12 

38 

14 

37 

33 

77 

b 

b 

237 

Licenses 

2 

3 

4 

83 

16 

8 

5 

145 

35 

2 

303 

Commercial  products 

3 

2 

1 

7 

0 

1 

7 

8 

5 

2 

36 

Commercial  processes 

3 

1 

2 

6 

0 

1 

3 

12 

1 

0 

29 

Spin-off  company 

1 

0 

0 

1 

1 

0 

2 

2 

b 

b 

7 

Subtotal 

98 

49 

59 

179 

180 

193 

273 

1,316 

131 

32 

2,510 

Other 

New  program 

20 

8 

3 

0 

2 

6 

29 

91 

b 

b 

159 

Invention  disclosures 

43 

32 

54 

122 

54 

89 

11 

212 

b 

b 

617 

Other0 

141 

0 

4 

0 

1 

96 

15 

0 

101 

165 

523 

Subtotal 

204 

40 

61 

122 

57 

191 

55 

303 

101 

165 

1,299 

Total 

2,321 

3,407 

3,882 

1,062 

904 

3,065 

3,948 

4,159 

1,929 

725 

25,402 

Legend 


ANL  =  Argonne  National  Laboratory 

LBL  =  Lawrence  Berkeley  Laboratory 

ORNL  =  Oak  Ridge  National  Laboratory 

PNL  =  Pacific  Northwest  Laboratory 

INEL  =  Idaho  National  Engineering  Laboratory 

LLNL  =  Lawrence  Livermore  National  Laboratory 

LANL  =  Los  Alamos  National  Laboratory 

SNL  =  Sandia  National  Laboratories 

BNL  =  Brookhaven  National  Laboratory 

NREL  =  National  Renewable  Energy  Laboratory 

aFiscal  year  1992. 

bResponses  were  not  collected  from  Brookhaven  National  Laboratory  and  National  Renewable 
Energy  Laboratory. 

°Research  subprogram  outputs  other  than  those  listed  above,  such  as  technical  abstracts, 
workshops  for  laboratory  users,  and  an  electronic  bulletin  board  service. 
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Table  V.2:  National  Laboratories’  Research  Program  Outputs  by  Program8 


Outputsb 

ER 

CE 

ES&H 

NE 

DP 

NPR 

ERWM 

FE 

CRWM 

PPA 

INT 

WFO 

Total 

Publications  and  reports 

Books 

46 

12 

0 

3 

23 

0 

1 

0 

0 

0 

0 

8 

93 

Articles 

3,951 

475 

5 

33 

1,040 

0 

45 

12 

32 

2 

0 

666 

6,261 

Book  chapters0 

257 

35 

0 

2 

39 

0 

4 

5 

1 

0 

0 

17 

360 

Reports 

1,692 

582 

20 

393 

2,407 

80 

296 

50 

84 

16 

0 

1,012 

6,632 

Conference  papers 

3,463 

663 

6 

219 

2,022 

9 

215 

44 

70 

2 

0 

773 

7,486 

Subtotal 

9,409 

1,767 

31 

650 

5,531 

89 

561 

111 

187 

20 

0 

2,476 

20,832 

Outputs  related  to 

commercial  product 

development 

Prototypes 

85 

39 

0 

20 

309 

0 

18 

1 

0 

0 

0 

63 

535 

Algorithms 

43 

16 

0 

10 

671 

0 

6 

1 

2 

0 

0 

19 

768 

Software 

49 

17 

0 

29 

233 

0 

17 

3 

0 

3 

0 

200 

551 

Patents0 

52 

18 

0 

5 

111 

0 

6 

0 

0 

0 

0 

7 

199 

Licenses 

41 

22 

0 

0 

152 

0 

1 

0 

0 

0 

0 

4 

220 

Commercial  products 

3 

7 

0 

1 

13 

0 

1 

0 

0 

0 

0 

4 

29 

Commercial  processes 

3 

4 

0 

1 

13 

0 

1 

0 

0 

0 

0 

1 

23 

Spin-off  company0 

3 

0 

0 

0 

3 

0 

0 

0 

0 

0 

0 

0 

6 

Subtotal 

279 

123 

0 

66 

1,505 

0 

50 

5 

2 

3 

IBa] 

298 

2,331 

Other 

New  program0 

25 

12 

0 

2 

97 

0 

5 

2 

0 

0 

0 

16 

159 

Invention  disclosures0 

80 

58 

0 

28 

270 

0 

40 

0 

0 

0 

0 

19 

495 

Otherd 

139 

168 

90 

0 

108 

0 

11 

6 

0 

0 

0 

1 

523 

Subtotal 

244 

238 

90 

30 

475 

0 

56 

8 

36 

1,177 

Total 

9,932 

2,128 

121 

746 

7,511 

89 

667 

124 

189 

23 

0 

2,810 

24,340 

(Table  notes  on  next  page) 
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Legend 

ER  =  Energy  Research 

CE  =  Conservation  and  Renewable  Energy 

ES&H  =  Environment,  Safety  and  Health 

NE  =  Nuclear  Energy 

DP  =  Defense  Programs 

NPR  =  New  Production  Reactors 

ERWM  =  Environmental  Restoration  and  Waste  Management 
FE  =  Fossil  Energy 

CRWM  =  Civilian  Radioactive  Waste  Management 
PPA  =  Policy  Planning  and  Analysis 
INT  =  Intelligence 
WFO  =  Work  for  Others 

aFiscal  year  1992. 

bPacific  Northwest  Laboratory  provided  information  about  all  outputs  for  the  laboratory  as  a  whole 
that  are  not  included  in  the  data  presented  here. 

cResponses  were  not  collected  from  Brookhaven  National  Laboratory  and  National  Renewable 
Energy  Laboratory. 

dResearch  subprogram  outputs  other  than  those  listed  above,  such  as  technical  abstracts, 
workshops  for  laboratory  users,  and  an  electronic  bulletin  board  service. 
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Table  V.3:  Proportion  of  Funds  Spent 
by  Programs  With  More  Than  100 
Outputs  Related  to  Commercial 
Product  Development8 


R&D-related  activity 

ER 

CE 

DP 

WFO 

Basic  research 

53.1 

4.4 

7.2 

11.5 

Applied  research 

15.2 

25.9 

32.5 

26.1 

Development 

10.1 

28.6 

40.9 

23.4 

Technical  assistance  to 

Government  agencies 

4.0 

5.9 

9.7 

25.1 

Universities 

6.5 

4.0 

0.7 

2.8 

Private  firms  or  industrial  organizations 

2.8 

11.6 

0.9 

2.2 

Technology  transfer  to 

Government  agencies 

4.1 

3.1 

2.8 

4.3 

Private  firms  or  industrial  organizations 

2.5 

15.7 

2.5 

4.4 

Otherb 

1.7 

0.7 

2.8 

0.4 

Total 

100 

100 

100 

100 

Legend 

ER  =  Energy  Research 

CE  =  Conservation  and  Renewable  Energy 

DP  =  Defense  Programs 

WFO  =  Work  for  Others 


aFiscal  year  1992. 

bSubprogram  expenditures  for  activities  other  than  those  listed  above,  such  as  training  graduate 
students  and  postdoctoral  fellows  or  safety  procedures. 
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Figure  V.1:  Trends  in  Outputs  Related 
to  Commercial  Product  Development3 
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aThe  number  of  outputs  is  shown  only  for  research  subprograms  in  the  study  population  that  were 
in  operation  all  4  years  (fiscal  years  1989-92). 
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Data  concerning  the  formation  of  cooperative  research  and  development 
agreements,  expenditures  for  r&d  in  critical  technologies,  and  the  views  of 
national  laboratory  program  managers  on  their  programs’  potential  for 
commercial  product  development  are  presented  below. 


Table  VI.1:  CRADAs  in  Effect  at  the  National  Laboratories® 

1992 

1991 

1990 

1989 

National  laboratory6 

All  4 
years' 

Any  4 
years6 

All  4 

years' 

Any  4 
years6 

All  4 

years' 

Any  4 
years6 

All  4 
years' 

Any  4 
years6 

Argonne 

7 

7 

2 

2 

0 

0 

0 

0 

Brookhaven 

4 

4 

0 

0 

0 

0 

0 

0 

Lawrence  Berkeley 

4 

4 

0 

0 

0 

0 

0 

0 

Oak  Ridge 

55 

55 

38 

38 

20 

20 

9 

9 

Pacific  Northwest 

4 

9 

0 

0 

0 

0 

0 

0 

Idaho 

12 

12 

9 

9 

4 

4 

1 

1 

Lawrence  Livermore 

13 

13 

3 

3 

1 

1 

1 

1 

Los  Alamos 

18 

18 

8 

8 

6 

6 

6 

6 

Sandia 

13 

74e 

4 

11 

0 

0 

0 

0 

Total 

130 

196 

64 

71 

31 

31 

17 

17 

“Fiscal  years  1989-92. 


bResponses  on  CRADA  formation  were  not  collected  from  National  Renewable  Energy 
Laboratory. 

cThese  are  CRADAs  formed  for  research  sponsored  by  subprograms  that  were  in  operation  every 
year  during  fiscal  years  1989-92. 

dThese  are  CRADAs  formed  for  research  sponsored  by  subprograms  that  were  initiated  in  any 
year  during  fiscal  years  1989-92. 

'Most  of  the  CRADAs  formed  in  fiscal  year  1992  were  sponsored  by  the  DOE  defense  program 
technology  transfer  initiative  at  Sandia.  This  subprogram  was  initiated  in  June  1990  to  identify 
opportunities  for  commercializing  technologies  produced  by  DOE-funded  defense  research 
activities. 
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Table  VI.2:  CRADAs  in  Effect  Within  the  National  Laboratories  by  Research  Program8 


Research  program11 _ 

Energy  Research _ 

Conservation  and  Renewable  Energy _ 

Environment,  Safety  and  Health _ 

Nuclear  Energy _ 

Defense  Programs _ 

New  Production  Reactors _ 

Environmental  Restoration  and  Waste  Management 

Fossil  Energy _ _ 

Civilian  Radioactive  Waste  Management 

Policy  Planning  and  Analysis _ 

Intelligence _ _ 

Work  for  Others 


All  4  Any  4  All  4  Any  4  All  4  Any  4  All  4  Any  4 

years'  years'1  years0  years'*  years0  years'*  years0  years'1 


"Fiscal  years  1989-92. 

bResponses  were  not  collected  from  National  Renewable  Energy  Laboratory. 

"These  are  CRADAs  formed  for  research  sponsored  by  subprograms  that  were  in  operation  every 
year  during  fiscal  years  1989-92. 

"These  are  CRADAs  formed  for  research  sponsored  by  subprograms  that  were  initiated  in  any 
year  during  fiscal  years  1989-92. 

eMost  of  the  CRADAs  formed  in  fiscal  year  1992  were  sponsored  by  the  DOE  defense  program 
technology  transfer  initiative  at  Sandia.  This  subprogram  was  initiated  in  June  1990  to  identify 
opportunities  for  commercializing  technologies  produced  by  DOE-funded  defense  research 
activities. 
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Table  VI.3:  Mean  Percent  Expenditures  for  Critical  Technologies  Within  Laboratories8 


Critical  technology11 

ANL 

LBL 

ORNL 

PNL 

INEL 

LLNL 

LANL 

SNL 

All  labs 

Material  synthesis  and  processing 

4.3 

5.5 

11.8 

4.3 

4.6 

3.6 

13.0 

1.9 

6.2 

Electronic  and  photonic  materials 

1.1 

4.4 

1.7 

0.2 

0 

1.9 

1.4 

2.1 

1.7 

Ceramics 

2.2 

2.2 

9.8 

0.5 

0.8 

0.7 

0.7 

0.4 

1.8 

Composites 

0.2 

0.4 

4.0 

0.6 

6.9 

0.4 

0.9 

0.2 

1.0 

High-performance  metals  and  alloys 

3.0 

1.5 

8.9 

0 

1.9 

1.6 

1.2 

1.4 

2.2 

Flexible  computer  integrated 
manufacturing 

0.2 

0 

1.6 

0.7 

1.3 

0 

0.6 

0.9 

0.6 

Intelligent  processing  equipment 

0.2 

0 

0.3 

1.1 

3.7 

0.3 

1.1 

0.8 

0.7 

Micro-  and  nanofabrication 

0.2 

0.4 

1.5 

0.1 

0.5 

0.4 

1.2 

0.7 

0.7 

System  management  technologies 

0 

0.2 

0.6 

5.0 

0.8 

0.8 

0.7 

1.4 

1.2 

Software 

2.2 

0.7 

0.4 

2.4 

12.0 

6.1 

3.5 

5.6 

4.0 

Microelectronics  and  optoelectronics 

0.6 

1.1 

0.5 

2.3 

0.3 

2.1 

2.3 

11.5 

3.7 

High-performance  computing  and 
networking 

1.9 

0.5 

3.3 

0.2 

3.4 

5.0 

2.0 

1.2 

2.4 

High-definition  imaging  and  displays 

0.4 

1.0 

0.2 

1.1 

1.2 

0.3 

0.5 

0.1 

0.4 

Sensors  and  signal  processing 

0.8 

0.2 

1.5 

3.5 

7.1 

5.1 

4.2 

9.0 

4.6 

Data  storage  and  peripherals 

0.1 

0.2 

0 

0.8 

0 

0.4 

1.1 

1.9 

0.8 

Computer  simulation  and  modeling 

5.7 

1.4 

5.0 

5.4 

14.6 

12.7 

4.5 

5.0 

6.7 

Applied  molecular  biology 

0.2 

8.6 

4.6 

0.7 

1.4 

2.4 

1.6 

0 

1.9 

Medical  technology 

0.8 

4.7 

0.6 

0.3 

0.1 

0.3 

0.4 

0 

0.6 

Aeronautics 

0 

0 

0.3 

1.0 

0.5 

0 

0 

0 

0.2 

Surface  transportation  technologies 

1.9 

0 

1.2 

0 

2.2 

0 

0 

0.4 

0.5 

Energy  technologies 

32.0 

24.2 

19.7 

7.6 

24.8 

18.7 

4.5 

3.5 

13.6 

Pollution  minimization,  remediation,  and 
waste  management 

18.8 

9.1 

1.9 

41.3 

7.5 

3.8 

3.8 

1.9 

8.8 

Other0 

5.3 

0.1 

6.6 

15.3 

4.3 

32.9 

2.1 

0.4 

10.2 

Research  funds  not  expended  on 
critical  technologies 

18.0 

33.3 

14.1 

5.6 

0 

0.5 

48.5 

49.9 

25.9 

Total 

100 

100 

100 

100 

100 

100 

100 

100 

100 

(Table  notes  on  next  page) 
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LEGEND 

ANL  =  Argonne  National  Laboratory 

LBL  =  Lawrence  Berkeley  Laboratory 

ORNL  =  Oak  Ridge  National  Laboratory 

PNL  =  Pacific  Northwest  Laboratory 

INEL  =  Idaho  National  Engineering  Laboratory 

LLNL  =  Lawrence  Livermore  National  Laboratory 

LANL  =  Los  Alamos  National  Laboratory 

SNL  =  Sandia  National  Laboratories 


“Fiscal  year  1992. 

bResponses  were  not  collected  from  Brookhaven  National  and  National  Renewable  Energy 
Laboratory. 

“Subprogram  expenditures  for  activities  other  than  those  listed  above,  such  as  robotics,  special 
nuclear  materials,  environmental  R&D,  and  detector  technology. 
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Table  VI.4:  Mean  Percent  Expenditures  for  Critical  Technologies  Within  Programs8 


Critical  technology11 

Material  synthesis  and 
processing 

Electronic  and  photonic 
materials 

Ceramics 


ER  CE  ES&HC  NE  DP  NPR  ERWM  FE  CRWM  PPA  INT  WFO  programs 


7.0  9.6 

3.1  0.6 

3.5  12.5 

~Z3  2J 


0.5  0.3 

0.6  0.4 

0.1  0.3 

2.8  0.9 

1.0  0.6 


Composites _ 2.3  2.7 

High-performance 

metals  and  alloys _ 4.0  2.5 

Flexible  computer 

integrated  manufacturing  0.4  0 

Intelligent  processing 

equipment _ 0-5  0.3 

Micro-  and 

nanofabrication  0.6  0.4 

System  management 

technologies  0.1  0.3 

Software  2.8  0.9 

Microelectronics  and 
optoelectronics _ 1.0  0.6 

High-performance 
computing  and 

networking _ 7J _ 0_ 

High-definition  imaging 

and  displays _ 0.4  0.1 

Sensors  and  signal 

processing _ 0.9  0.7 

Data  storage  and 

peripherals _ 0J _ 0_ 

Computer  simulation  and 

modeling _ 6.5  2.1 

Applied  molecular 

biology _ 8.8  0.3 

Medical  technology _ 2.2  0 

Aeronautics _ 0  0 

Surface  transportation 

technologies _ 0  4.2 

Energy  technologies _ 12.6  39.7 

Pollution  minimization, 

remediation,  and  waste 

management _ 2.3  1.7 

Otherd  10,5  0.4 


0.4  0.1 


0.9  0.7 


0.7  0 


8.8  0.3 

2.2  0 

0  0 

0  4.2 

1Z6  39.7 


0  6.1  7.3  0 

0  0.7  1.9  0 

0  2.4  0.5  3.1 

0  0,4  0.3  0 

0  6.2  1.2  4.6 

0  0.3  0.6  0 

0  0.3  0.8  0 

0  0  1.0  0 

0  0.7  0.9  0 

0  2.3  5.3  0 

0  0.7  6.7  0 


0  2.0  1.2  0 


0  0.3  0.4  4.6 


5.0  3.6  7.4 


0  1.2 


6.5  2.1  10.0  4.0  7.4  4.6 


0  0  0.1  0 

0  0  0  0 

0  0  0  0 

0  0  0.2  0 

~0  6a6  3l3  83.1 


1.6  8.5 

0  5.3 

0.1  9.8 

0,5  3.9 

0.2  3.8 


0.1  0 


0  0.6 


0.3  7.6 


0  0 


0  0 


2.9  2.4 


1.1  0 


3.9  8.5 


0.6  0 

0  0 

0  0 


1.2  0 
1.0  41.3 


4.4  0  0  3.1 

0  0  0  0.5 

“0  0  0  OiT 

0  0  0  1.7 

1.8  0  0  1.4 

0  0  0  1.8 

0  0  0  1.6 

0  0  0  0.7 

7.1  0  0  2.9 

5.6  5.0  0  5.6 

0  0  0  2.4 


0.3  0  0  1.6 

0  0  0  1.0 

0  0  10.0  3.3 

0  0  0  0 

12.2  5.0  0  8.4 

0  0  0  0 

0  0  0  1lT 

0  0  0  1.3 

0  0  0  1.0 

25.8  90.0  0  12.5 


2.3  1.7  10.0  1.3  2.5  0  77.6  0.3 


0  14.5 


0.5  8.1 


9.5  0  0  11.1 _ &8 

3.6  0  90.0  9.3  10.2 

(continued) 
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Critical  technology11 

ER 

CE 

ES&He 

NE 

DP 

NPR 

ERWM 

FE 

CRWM 

PPA 

INT 

WFO 

All 

programs 

Research  funds  not 
expended  on 
critical  technologies 

22.7 

20.3 

75.0 

0 

34.9 

0 

4.8 

0 

29.6 

0 

0 

27.1 

25.9 

Total 

100 

100 

100 

100 

100 

100 

100 

100 

100 

100 

100 

100 

100 

Legend 

ER  =  Energy  Research 

CE  =  Conservation  and  Renewable  Energy 

ES&H  =  Environment,  Safety  and  Health 

NE  =  Nuclear  Energy 

DP  =  Defense  Programs 

NPR  =  New  Production  Reactors 

ERWM  =  Environmental  Restoration  and  Waste  Management 
FE  =  Fossil  Energy 

CRWM  =  Civilian  Radioactive  Waste  Management 
PPA  =  Policy  Planning  and  Analysis 
I  NT  =  Intelligence 
WFO  =  Work  for  Others 

aFiscal  year  1992. 

bResponses  were  not  collected  from  Brookhaven  National  and  National  Renewable  Energy 
Laboratory. 

cOnly  one  laboratory  provided  complete  information  for  the  ES&H  program.  ES&H  activity  at  this 
laboratory  provides  support  for  environmental  protection  at  DOE  sites  in  several  areas,  ensuring 
safe  facility  management  practices,  developing  and  recommending  radiation  and  chemical 
protection  policies  and  practices,  and  evaluating  the  health  of  DOE  personnel  and  the  public. 
R&D  to  improve  dosimetry  and  measurement  techniques  also  is  undertaken. 

dSubprogram  expenditures  for  activities  other  than  those  listed  above,  such  as  robotics,  special 
nuclear  materials,  environmental  R&D,  and  detector  technology. 
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Figure  VI.1:  Program  Managers’  Views 
of  Programs’  Commercial  Product 
Potential3 


10.5% 

Long-term  future 

4.5% 

Very  long  term 


Immediate  future 


Short-term  future 


aTotal  exceeds  100  owing  to  rounding. 
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Note:  GAO  comments 


supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


See  comment  1 . 


See  comment  2. 


Department  of  Energy 

Washington,  DC  20585 
September  14,  1994 


Hr.  Kwai -Cheung  Chan 
Director,  Program  Evaluation  in  Physical 
Systems  Areas 

Program  Evaluation  and  Methodology 
Division 

U. S.  General  Accounting  Office 
Washington,  D.C.  20548 

Dear  Hr.  Chan: 

The  Department  of  Energy  appreciates  the  opportunity  to  review  and  comment  on 
the  General  Accounting  Office's  draft  report  entitled  "National  Laboratories: 
Research  and  Development  Activities." 

Hinor  editorial  changes  have  been  provided  to  the  General  Accounting  Office 
under  separate  cover. 

Our  major  concern  is  that  the  data  base,  definitions  of  research  and 
development,  and  conclusions  based  on  the  data  do  not  provide  an  accurate 
picture  of  the  current  efforts  of  the  Department  of  Energy  laboratories  and  do 
not  contribute  to  the  debate  on  the  role  of  the  laboratories  in  helping  to 
address  the  Nation's  security,  environmental,  energy,  scientific  and  economic 
needs  of  the  future.  The  report  reflects  a  lack  of  understanding  relating  to 
the  phases  of  research  and  development,  the  basic  and  applied  research  and 
development  activities  of  the  laboratories,  and  the  Department's  technology 
transfer  program. 

For  example,  the  report  combines  "development"  with  technology  transfer  and 
technical  assistance  and  defines  it  as  research  related  to  "commercial  product 
development."  This  leads  to  the  erroneous  conclusion  that  laboratory  research 
and  development  is  evenly  divided  between  basic  and  applied  research,  and 
research  related  to  commercial  product  development.  The  report  notes, 
however,  most  of  the  development  work  was  devoted  to  defense  research  "which 
may  have  more  limited  market  opportunities,  rather  than  nondefense  research." 

Horeover,  additional  data  is  necessary  to  help  Congress  consider  how  the 
National  Laboratories  of  the  Department  of  Energy  can  best  be  focused  to  help 
solve  the  problems  our  Nation  faces  during  the  current  decade.  That  data 
could  include  quantitative  trends  on  how  the  missions  of  the  Department  and 
its  laboratories  evolved,  information  on  the  core  competencies  developed  at 
the  laboratories  and  dollars  invested  in  the  extensive  science  base.  Budget 
data  on  current  missions  of  the  laboratories  in  support  of  critical 
technologies,  and  data  on  all  of  the  laboratory  collaborations  with  industry 
that  contribute  to  the  Nation's  economic  growth  would  also  be  useful.  Also 
the  report  should  Include  more  detailed  information  leading  to  a  better 
understanding  of  work  carried  out  by  the  laboratories  for  other  agencies  and 
the  private  sector,  and  their  impact  on  the  economy. 


Printed  whh  soy  ink  on  recycled  paper 
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See  comment  3. 


See  comment  4. 


See  comment  5. 


See  comment  6. 


See  comment  7. 
See  comment  8. 


See  comment  9. 


It  is  misleading  to  say  that  the  National  Laboratories  do  product  development 
work.  What  they  typically  do  is  work  with  partners  so  that  the  partners  can 
then  carry  out  the  product  or  process  development  work.  The  statements  made 
in  the  draft  about  product  development  do  not  seem  to  be  based  on  a  firm 
understanding  of  the  product  development  process.  This  is  further  revealed  in 
the  draft  report  statement  about  lack  of  outcomes  for  the  "development”  work 
done  at  the  laboratories.  It  typically  will  take  years  from  the  conclusion  of 
a  Cooperative  Research  and  Development  Agreement  (CRADA)  and  the  transfer  of  a 
technology  to  a  partner,  to  the  commercialization  of  a  product.  Incremental 
product  and  process  improvements  will  proceed  faster.  The  draft  does  not 
recognize  this. 

The  report  states  that  the  General  Accounting  Office  could  find  no  existing 
documentation  to  develop  a  comprehensive  description  of  current  research 
activities  in  the  ten  laboratories,  and  therefore,  collected  the  data  through 
a  survey  of  the  laboratories.  The  institutional  plans  and  research  and 
development  data  base  for  all  of  the  laboratories  contain  data  comparable  to 
and  more  up  to  date  than  the  data  in  the  report  and  is  readily  available  in 
the  Department  of  Energy  Headquarters.  One  example  of  the  report's  inaccurate 
data  is  that  Department  of  Energy  now  has  more  than  1,000  CRADAs  rather  than 
the  183  cited.  The  absence  of  the  General  Accounting  Office's  interaction 
with  Headquarters  also  has  resulted  in  a  lack  of  understanding  of  the  missions 
of  the  Department  and  the  roles  of  the  laboratories  in  carrying  them  out.  The 
report  does  not  clearly  state  that  funds  are  distributed  among  the 
laboratories  based  on  program  needs,  laboratory  expertise,  and  peer  review 
mechanisms.  The  report  also  suffers  from  an  over-emphasis  on  the  number  of 
CRADAs  at  the  laboratories  and  a  lack  of  discussion  of  other  mechanisms,  such 
as  cost-shared  cooperative  agreements,  scientific  user  facilities,  licensing 
agreements,  and  personnel  exchange  agreements.  The  report's  attempt  to 
quantify  and  compare  outputs  from  the  laboratories  is  not  a  valid  method  for 
assessing  the  impact  of  these  institutions.  In  this  regard,  the  General 
Accounting  Office's  finding  that  the  laboratories  have  produced  more 
publications  and  reports  than  they  did  commercial  products  is  a  fallacious 
comparison  that  implies  an  unwarranted  criticism  of  the  research  enterprise. 

Also,  the  report  does  not  contain  human  resources  data,  as  requested.  The 
numbers  of  scientists,  engineers  and  skilled  technicians  could  be  compared 
with  the  research  and  development  resources  of  other  agencies  and  the  private 
sector.  In  addition,  the  use  of  percentages  to  describe  laboratory  research 
and  development  expenditures  does  not  give  a  true  picture  of  the  dollar 
magnitude  of  the  investment  in  the  research  and  development  areas  discussed, 
nor  can  they  be  compared  to  the  investments  of  others.  A  comparison  of  the 
research  and  development  activities  of  the  laboratories  of  other  agencies  to 
the  Department's  laboratories  would  also  be  useful  in  determining  whether  the 
enormous  technical  capabilities  and  experimental  facilities  will,  perhaps, 
become  the  best  means  of  enabling  new  national  research  and  development 
missions.  It  will  be  very  difficult,  if  not  impossible,  to  make  such 
comparisons  with  the  General  Accounting  Office  data  base. 
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The  following  are  gao’s  comments  on  the  September  14, 1994,  letter  from 

DOE. 


GAO  Comments 


1.  The  definitions  for  basic  research,  applied  research,  and  development 
that  our  study  employs  are  derived  from  a  Congressional  Budget  Office 
study  of  the  federal  r&d  enterprise.  The  definition  of  technology  transfer  is 
the  one  used  in  our  study  of  the  Technology  Transfer  Act  of  1986  and  the 
definition  of  technical  assistance  is  based  on  expert  opinion.  Our  analysis 
examined  the  laboratories’  effort  in  each  type  of  activity  separately,  and 
grouped,  in  two  major  categories,  in  order  to  address  the  study  objective: 
to  provide  an  empirical  base  for  examining  the  extent  to  which  the 
laboratories  are  engaged  in  basic  and  applied  research  or  research  related 
to  commercial  product  development.  Figure  1  and  tables  IV.1-IV.4  allow 
the  reader  to  view  our  findings  in  both  the  two  nuyor  categories  and  as 
separate  R&D-related  activities.  The  finding  for  each  major  category 
presented  in  figure  1  is  the  sum  of  the  findings  for  the  corresponding 
separate  R&D-related  activities  presented  in  the  last  column  of  table  IV.  1. 

doe  disagrees  with  the  category  we  established  for  “research  related  to 
commercial  product  development”— that  is,  that  development,  technical 
assistance,  and  technology  transfer  are  all  laboratory  activities  related  to 
commercial  product  development — but  does  not  question  our  definitions 
or  findings  for  each  separate  activity.  We  agree  that  doe  may  decline  to 
accept  our  definition  for  research  related  to  commercial  product 
development,  but  we  do  not  agree  that  our  finding  for  the  sum  of  the  three 
separate  activities  is  erroneous.  This  finding  is  based  on  laboratory 
research  managers’  estimates  of  the  distribution  of  their  subprograms’ 
expenditures  that  were  collected,  verified,  and  analyzed  according  to 
generally  accepted  government  auditing  standards.  We  consider  these 
estimates,  made  by  research  managers  who  are  closely  involved  with  the 
r&d,  more  accurate  than  estimates  that  may  be  obtained  by  other  methods. 

2.  The  analyses  we  produced  were  intended  to  establish  baseline  data  for 
addressing  empirical  questions  underlying  the  public  debate,  rather  than 
to  serve  as  a  comprehensive  analysis  of  the  laboratories’  roles.  To  address 
the  study  objective,  we  focused  on  the  10  laboratories  as  a  set  of 
institutions,  on  comparing  the  distribution  of  expenditures  for  five  types  of 
R&D-related  activities  both  within  and  among  the  10  laboratories,  on  the 
nature  and  scope  of  their  outputs,  and  on  their  potential  for  working  with 
industry  to  bring  commercial  products  to  market.  Given  this  approach, 
with  the  exception  of  expenditures  for  critical  technologies  and 
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collaboration  with  industry,  which  we  do  examine,  the  other  factors  doe 
suggests  for  analysis  are  beyond  the  scope  of  this  study.  However,  we 
anticipate  that  our  study  might  stimulate  another  party  to  undertake  the 
type  of  institutional,  comparative  analysis  that  doe  suggests. 

3.  We  agree  with  doe  that  the  laboratories  collaborate  in  r&d  with  industry 
partners  who  then  perform  the  additional  testing  and  research  activities 
required  for  commercial  application.  We  also  agree  that  “it  typically  should 
take  years  from  the  conclusion  of  a  crada  and  the  transfer  of  a  technology 
to  a  partner,  to  the  commercialization  of  a  product.”  The  explicit 
definitions  of  terms  and  the  discussion  of  cradas  in  the  report  make  this 
clear.  (See  pp.  6, 12,  and  15.)  However,  we  disagree  that  the  report 
attributes  commercial  product  development  work  to  the  national 
laboratories. 

4.  We  state  in  the  section  on  Methodology  that  we  began  our  work  with  a 
survey  of  the  laboratories’  r&d  activities  because  we  could  find  no 
sufficiently  comprehensive  (emphasis  added)  existing  documentation. 
To  confirm  that  we  had  not  overlooked  an  important  information  source 
when  we  designed  and  implemented  our  data  collection  strategy,  we  made 
inquiries  about  doe’s  institutional  plan  and  research  and  development 
databases.  We  found  that  doe  headquarters  maintains  only  the  institutional 
plan  database  and  that  it  includes  only  one  of  the  data  items,  research 
program  budget,  that  we  used  in  our  report.  This  budget  information  was 
available  for  fiscal  years  1989-91  when  we  implemented  our  survey  but 
would  not  have  been  useful  for  our  analyses  because  it  is  not  compiled  at 
the  same  level  of  detail  as  our  data 

We  also  found  that  the  research  and  development  database  is  not  one  of 
doe’s  databases.  It  is  being  developed  by  the  Critical  Technologies 
Institute  for  the  Office  of  Science  and  Technology  Policy  (ostp)  in  the 
Executive  Office  of  the  President.  When  it  is  complete,  it  will  have  five 
data  items  analogous  to  our  data  However,  this  database  was  not 
available  when  we  developed  our  national  laboratory  inventory  and  is  not 
now  available  to  users  other  than  ostp.  Forty-one  of  the  items  in  our  report 
are  not  included  in  either  the  institutional  plan  or  research  and 
development  databases. 

The  Laboratory  Management  Division  in  doe’s  Office  of  Energy  Research 
maintains  the  institutional  plan  database.  It  has  research  program  budget 
data  for  fiscal  years  1979  to  the  present  at  the  program  level  for  9  of  the  10 
national  laboratories,  and  it  has  subprogram  budget  data  for  selected 
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programs,  such  as  energy  research,  defense  programs,  civilian  radioactive 
waste  management,  and  work  for  others.  Because  they  are  incomplete  at 
the  subprogram  level,  these  data  would  not  have  been  useful  for  our 
R&D-related  activities  and  critical  technologies  analyses,  which  required 
budget  data  for  all  subprograms  in  our  sample.  Further,  none  of  the  budget 
data  for  the  National  Renewable  Energy  Laboratory  are  included  in  the 
institutional  plan  database.  These  data  must  be  obtained  from  NREL’s 
hardcopy  institutional  plan,  which  is  available  from  the  Office  of  Energy 
Efficiency  and  Renewable  Energy  at  headquarters. 

The  Critical  Technologies  Institute’s  research  and  development  database 
will  have  information  on  laboratory  expenditures  for  basic  research, 
applied  research,  development,  and  technology  transfer  for  research 
subprogram  categories  analogous,  but  not  identical,  to  those  we  used,  and 
on  cradas — for  the  national  laboratories  as  well  as  for  the  laboratories  of 
several  other  federal  agencies — when  it  is  available  to  organizations  other 
than  ostp.  The  Critical  Technologies  Institute  representative  to  whom  we 
spoke  could  not  specify  when  the  database  will  be  available.  However,  the 
research  and  development  database  will  not  have  information  comparable 
to  the  16  research  subprogram  outputs  we  collected  from  the  laboratories 
nor  on  the  proportion  of  subprogram  expenditures  for  the  22  critical 
technologies  and  the  proportion  of  expenditures  for  technical  assistance. 

We  are  also  aware  that  abstracts  of  crada  agreements  can  be  obtained 
through  doe  headquarters  from  the  Office  of  Scientific  and  Technical 
Information,  which  is  based  in  Oak  Ridge,  Tennessee.  However,  we  also 
found  these  data  to  be  incomplete.  In  August  1992,  we  requested  these 
data  through  doe’s  Office  of  Technology  Utilization  at  headquarters  and 
received  147  abstracts  for  the  nine  laboratories  from  which  we  collected 
crada  information — 49  fewer  than  the  total  the  laboratories  reported  to  us. 
Since  the  fiscal  year  was  not  then  complete,  we  assumed  that  all  crada 
information  had  not  yet  been  reported  to  doe  or  entered  into  the  database. 

Our  experience  developing  the  survey  frame,  moreover,  suggested  that  the 
laboratories’  institutional  plan  data  needed  modification  to  address  our 
study  requirements  and  that  the  information  available  from  doe  was  not 
consistent  with  information  available  from  the  laboratories.  We  used  the 
list  of  research  programs  included  in  the  institutional  plans  as  a 
preliminary  frame  for  part  I  of  the  survey.  Recognizing  that  the 
laboratories  are  dynamic  institutions,  we  asked  each  laboratory  to  confirm 
the  list  before  survey  implementation.  Most  of  the  laboratories  made  both 
deletions  and  additions  to  the  list  to  meet  our  survey  selection  criteria. 
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(See  p.  4.)  We  used  the  lists  of  facilities  reported  in  the  doe  report,  Capsule 
Review  of  doe  Research  and  Development  Laboratories  and  Field 
Facilities,  as  a  preliminary  frame  for  part  II  of  the  survey.  The  laboratories 
made  deletions  and  additions  to  these  lists  as  well  and  in  two  cases  almost 
completely  replaced  them.  Changes  of  this  magnitude  confirmed  the 
strategy  of  collecting  data  directly  from  the  laboratories  to  address  our 
study’s  information  requirements. 

5.  During  the  agency  review  of  our  draft  report,  two  laboratories  provided 
us  with  additional  crada  information,  bringing  the  total  number  of  cradas 
in  effect  among  all  programs  in  operation  in  any  year  from  fiscal  year  1989 
to  1992  to  196.  (See  table  VI.  1.)  This  total  is  the  number  of  cradas  in  effect 
in  fiscal  year  1992,  rather  than  “now,”  to  which  doe  refers  and  which  we 
assume  is  fiscal  year  1994.  Moreover,  we  found  a  substantial  increase  in 
crada  formation  in  fiscal  year  1992,  sponsored  by  doe’s  defense  program 
technology  transfer  initiative  at  Sandia.  (See  tables  VI.  1  and  VI.2.)  It  is 
possible  that  the  increase  we  found  persisted  and  included  more 
laboratories,  bringing  the  total  to  1,000  in  fiscal  year  1994.  However,  such  a 
change  would  not  render  our  finding  for  fiscal  year  1992  inaccurate. 

6.  Brookhaven  brought  it  to  our  attention  that  the  number  of  cradas 
formed  is  limited  by  the  amount  of  money  allocated  to  a  laboratory  and 
that  this  amount  varies  widely  from  laboratory  to  laboratory.  We  agree 
with  Brookhaven  that  characterizing  crada  formation  as  the  “strongest” 
indicator  of  a  laboratory’s  commercial  product  potential  is  misleading  for 
this  reason,  and  we  have  modified  our  discussion  of  crada  findings. 

Scientific  user  facilities  and  personnel  exchanges  will  be  examined  in  a 
separate  study.  Licensing  is  described  in  the  section  on  Principal  Findings 
of  this  report.  (See  pp.  13-14.)  cradas  are  cost-shared  cooperative 
agreements  targeted  to  a  commercial  innovation. 

7.  We  treat  laboratory  outputs  as  measures  of  activity,  not  as  measures  of 
impact  or  productivity.  (See  pp.  7-8.) 

8.  We  found  that  the  10  laboratories  produced  many  more  publications  and 
reports  (21,593)  than  they  did  outputs  related  to  commercial  product 
development  (2,510)  in  fiscal  year  1992.  This  is  a  statement  of  fact, 
tabulated  from  reports  to  us  by  the  laboratories’  research  managers.  It 
describes  the  laboratories’  activity.  It  is  not  intended  as  a  criticism  of  the 
research  enterprise. 
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9.  The  purpose  of  this  report  was  to  examine  the  balance  of  R&D-related 
activity  across  the  laboratories,  rather  than  to  examine  the  magnitude  of 
the  r&d  investment.  We  used  the  proportion  of  funds  expended  for  each 
type  of  R&D-related  activity  as  a  measure  of  activity,  not  as  a  measure  of 
investment.  (See  pp.  6-7.)  An  examination  of  human  resources  and  a 
comparison  of  doe’s  national  laboratories  to  those  of  other  agencies  was 
beyond  the  scope  of  this  study,  given  its  focus  on  laboratory  R&D-related 
activity. 
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A  representative  of  Argonne,  Internal  Audit,  called  us  on  July  5, 1994,  to 
report  that  the  laboratory  had  no  substantive  comments  on  the  report 
draft. 


Page  75 


GAO/PEMD-95-2  National  Laboratories’  R&D  Activities 


Appendix  IX 


Comments  From  Brookhaven  National 
Laboratory 


Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


Now  p.  4. 

See  comment  1. 


Now  p.  7. 

See  comment  2. 


Now  table  IV.1,  p.  50. 
See  comment  3. 


BROOKHAVEN  NATIONAL  LABORATORY _ 

ASSOCIATED  UNIVERSITIES  INC  !-’  0  Box  5000 

Up  ion  New  York  i9/,vt>00u 

Tfl  <5i6)?S:>  2772 
EAx  (fjlA.i  2f?  5803 

office  of  the  Director  h  VtAH 

June  27,  1994 


Mr.  Kwai-Cheung  Chan,  Director 
Program  Evaluation  in  Physical  Systems  Area 
U.  S.  General  Accounting  Office 
Washington,  DC  20548 

Dear  Mr.  Chan: 

This  is  in  response  to  your  letter  dated  June  2,  1994  in  which  you  request  comments  to  the 
draft  report  entitled  “National  Laboratories:  Research  and  Development  Activities”.  Based  upon 
discussions  between  Ms.  Sara  Edmondson  of  your  office  and  Mr.  Anthony  Romano  of  my  office, 
most  of  our  initial  questions  about  the  report  have  been  resolved.  We  believe  that  your 
consideration  of  these  suggestions  will  more  accurately  reflect  the  Brookhaven  National 
Laboratory  contribution  to  your  report. 

The  following  comments  have  been  discussed  with  Sara: 

l  p  5,  We  believe  the  report  should  emphasize  the  fact  that  the  BNL  and  NREL  data  in  the 
report  are  based  upon  the  earlier  survey  instrument  which  is  somewhat  different  from  the 
one  included  in  the  report. 

2.  p.  10,  Table  2.  It  is  our  understanding  that  the  value  for  “Total  Budget”  of  S472.6M  is 
from  our  Institutional  Plan  dated  November  1990.  That  plan  shows  a  combined  value  of 
S365.7M  for  DOE  Effort,  WFO  and  Capital  Equipment  (Table  IX,  p.  99).  As  discussed 
with  Mr.  Romano  an  alternate  acceptable  value  could  be  derived  by  adding  the  individual 
B&R  budgets  in  the  survey  instrument  supplied  to  GAO  ($289. 1M)  adjusted  to  include 
the  appropriate  capital  equipment  budgets  from  Table  A.  1  attached  to  this  letter  and  by 
subtracting  the  budgets  of  those  programs  which  you  have  determined  do  not  fit  your 
R&D  program  definition.  The  work  papers  backing  up  this  request  are  attached  as  Exhibit 

A. 

3.  pl6.  Table  3 .  We  now  believe  the  value  for  BNL  in  this  table  was  highly  skewed  by  an 
erroneous  interpretation  on  the  part  of  one  of  our  program  managers.  He  assumed 
operation  of  the  NSLS  should  be  categorized  as  “other”  whereas  it  should  have  been  basic 
research.  Our  new  interpretation  is  attached  as  Exhibit  B. 

mt>  6S&2SI63NlOO£  CABI.I:  BROOK!  Afi'-rrMlW 
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Mr.  Kwai-Cheung  Chan 


June  27, 1994 


Now  p.  14. 

See  comment  4. 


See  comment  5. 


Now  p.  28. 


4.  p.  24.  We  question  the  statement  regarding  the  importance  of  the  formation  of  CRADAs 
as  the  strongest  indicator  of  potential  for  commercial  product  development.  The  number 
of  CRADAs  formed  is  limited  by  the  amount  of  money  allocated  to  a  laboratory  not  by  the 
number  of  good  CRADA  proposals.  The  amount  of  hinds  available  for  CRADA's  varies 
widely  from  laboratory  to  laboratory.  The  number  of  good  proposals  greatly  exceeds  the 
existing  funding  levels.  Certainly  the  formation  of  CRADAs  should  be  one  of  the  three 
indicators  of  the  national  laboratories'  potential  for  commercial  product  development,  but 
the  conclusion  that  it  is  the  “strongest"  indicator  is  misleading  since  it  is  dependent  upon 
the  funds  provided  by  DOE. 

5.  Table  VI.  1.  BNL  had  4  CRADAs  formed  in  F  which  are  not  reflected  this  table.  The  data 
had  been  provided  to  GAO  in  the  survey  instrument,  a  copy  of  which  is  attached  as  part  of 
Exhibit  C.  We  also  believe  this  should  be  reflected  in  Appendix  I,  Description  of  the 
National  Laboratories,  Brookhaven  National  Laboratory,  p.  6.  We  suggest  adding  the 
following  sentence  "Brookhaven  also  performs  technology  transfer  and  offers  science  and 
engineering  education  programs." 

Thank  you  for  the  opportunity  to  review  and  comment  on  the  report.  If  you  require  any 

additional  information  or  assistance,  please  call  Mr.  A.  Romano,  516-282-4024.  We  look 


forward  to  receiving  the  final  report. 


Very  truly  yours. 


N.  P.  Samibs  . 
Director 


Enclosures 


A  Romano 
H.  Grahn,  w/o  attach. 
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The  following  are  gao’s  comments  on  the  June  27, 1994,  letter  from 
Brookhaven  National  Laboratory. 


GAO  Comments 


1.  We  have  added  a  statement  to  the  report  clarifying  this  difference. 

2.  We  have  evaluated  the  data  Brookhaven  submitted  and,  after  making  the 
appropriate  changes,  added  it  to  the  database.  These  data  have  been 
incorporated  into  the  tables  included  in  the  report  letter  and  appendixes. 

3.  We  agree  with  Brookhaven’s  evaluation  of  this  response  and  have  made 
the  change  they  requested  to  the  database  and  report  tables. 

4.  We  agree  with  Brookhaven  and  have  modified  the  discussion  of  crada 
findings. 

5.  The  information  on  crada  formation  Brookhaven  submitted  in  the  pilot 
version  of  the  data  collection  instrument  has  been  added  to  the  database 
and  the  tables  in  appendix  VI.  We  also  have  added  the  sentence 
Brookhaven  suggests  to  appendix  I. 
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Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


See  comment  1 . 
See  comment  2. 


United  States  Government _ 

memorandum 


Department  of  Energy 
Idaho  Operations  Office 


DAf*  June  29,  1994 

subject:  Comments  of  GAO  Draft  Report  -  (CFO-PAD-THR-94-013) 


to:  Kwai-Cheung  Chan 

Director,  Program  Evaluation 
in  Physical  Systems  Areas 
General  Accounting  Office 

REFERENCE:  Letter,  K.  C.  Chan  to  A.  A.  Pitrolo,  Subject:  Draft  Report  -  National 
Laboratories:  Research  and  Development  Activities  (973356) 

ID  has  reviewed  the  draft  copy  of  the  GAO  report  prepared  from  responses  of  DOE's  10 
national  laboratories.  It  is  our  observation  that  interpretation  of  "R&D"  is  subjective  at  best. 
Because  of  this  subjectivity,  the  responses  from  the  10  national  laboratories  with  respect  to 
FY  1992 INEL  R&D  budgets  (Table  2)  should  not  be  utilized  for  purposes  of  comparison. 
ID's  interpretation  of  HQ  definitions/criteria  for  research  programs  resulted  in  our  not 
reflecting  the  entire  FY  1992  INEL  R&D  budget  on  Table  2.  Approximately  J176M  of  the 
S275M  FY  1992  INEL  R&D  budget  was  not  shown. 

Please  direct  any  questions  which  you  may  have  regarding  this  observation  and  comment  to 
Richard  Burger  (208-526-0536)  or  Jack  Reed  (208-526-0013). 
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The  following  are  gao’s  comments  on  the  June  29, 1994,  memorandum 
from  doe’s  Idaho  Operations  Office. 


GAO  Comments 


General  Comments  doe’s  Idaho  Operations  Office  representative,  who  responded  for  Idaho, 

observed  that  the  value  in  the  “r&d  Budget”  column  of  table  2  for  Idaho 
should  be  $275  million,  rather  than  $98.7  million  and  that  the  Idaho 
Operations  Office  made  this  determination  by  applying  doe  headquarters’ 
definitions  for  research  programs  to  Idaho’s  research  programs.  The  list  of 
Idaho  research  programs  to  which  the  Idaho  Operations  Office  applied  doe 
headquarters’  definitions  is  unspecified.  We  disagree  with  this 
determination,  because  it  violated  the  study  methodology. 


Specific  Comments  1.  The  r&d  budgets  of  the  10  national  laboratories  in  table  2  were  not 

compared. 

2.  We  coordinated  data  collection  from  the  laboratories  with  doe’s 
operations  office  representatives,  but  none  of  them  participated  in  any  of 
the  technical  activities  involving  survey  implementation.  Therefore,  the 
Idaho  Operations  Office  representative  may  not  have  been  aware  that  gao 
program  selection  criteria  should  have  been  employed  to  assess  the  “r&d 
Budget”  column  value  for  Idaho  in  table  2  to  be  consistent  with  the 
methodology  employed  for  the  other  nine  laboratories.  The  use  of  doe 
headquarters’  definitions  for  research  programs  to  make  this 
determination  would  result  in  a  list  of  subprograms  that  differs 
substantially  from  the  one  jointly  developed  by  gao  and  Idaho. 

Subprograms  included  in  the  survey  population  were  identified  by 
laboratory  representatives  who  applied  the  selection  criteria  we  specified 
(see  p.  4)  to  a  preliminary  subprogram  list  we  compiled  from  the 
institutional  plans  and  sent  to  the  laboratories.  This  approach  was 
followed  by  Idaho’s  representatives,  who  identified  10  subprograms.  We 
reduced  the  number  of  Idaho  subprograms  to  nine  during  the  editing  and 
coding  process.  The  $98. 7-million  value  in  the  “r&d  Budget”  column  is  the 
total  of  nine  research  subprogram  budgets  reported  by  Idaho  program 
managers  on  part  I  of  the  national  laboratory  inventory  data  collection 
instrument. 
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Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


See  comment  1 . 


See  comment  2. 


See  comment  3. 


Lawrence  Berkeley  Laboratory 

University  of  California  Berkeley,  California  94720 
Office  for  Planning  &  Development 
(510)486-4361 

June  17  1994 


Kwai-Cheung  Chan 

Director,  Program  Evaluation  in  Physical  Systems  Areas 
United  States  General  Accounting  Office 
Washington,  D.C.  20548 


Dear  Mr.  Chan: 

We  appreciate  the  opportunity  to  comment  on  the  GAO  Draft  Report  on  "National 
Laboratories:  Research  and  Development  Activities".  We  would  like  to  respond  both  with  some 
general  comments  on  the  role  of  basic  research  in  commercial  product  development  and  specific 
suggestions  on  particular  portions  of  the  document. 

The  general  comment  we  would  like  you  to  consider  relates  to  the  relationship  of  basic  and 
applied  research  to  commercial  development.  The  report  is  a  thoughtful  assessment  of  commercial 
product  development  in  the  research  and  development  activities  of  the  National  Laboratories.  What 
is  not  clear  is  the  relationship  of  commercial  product  development  to  the  broader  needs  of  industry 
or  the  nation.  These  broad  needs  include  the  scientific  and  technological  infrastructure  required  as 
a  precursor  to  any  industrial  technology  and  the  development  of  final  products.  As  one  example, 
no  industrial  development  would  be  possible  without  the  training  of  scientists  and  engineers  as 
supported  by  universities  and  the  national  laboratories,  yet  clearly  such  training  is  not  directed 
toward  a  particular  commercial  output. 

The  opening  paragraph  does  seem  to  raise  this  more  general  topic  by  addressing  "...interest 
in  the  current  balance  of  the  research  effort  in  the  10  laboratories  research  programs...  (and)  the 
extent  to  which  the  national  laboratories  are  engaged  now  in  basic  and  applied  research  or  in 
research  related  to  commercial  product  development."  This  rightly  implies  that  commercial  product 
development  is  a  subset  of  the  broader  research  and  development  role  of  the  national  laboratories. 
The  conclusion,  however,  deals  only  with  this  subset  Not  even  the  non-conmiercial  product 
output  of  nuclear  weapons  research  is  discussed  despite  being  one  of  the  major  activities.  In  short, 
we  believe  the  report  is  far  more  focused  than  what  is  implied  in  the  current  introduction  so 
clarification  and  discussion  of  this  point  would  be  valuable  to  all  parties. 

In  addition,  some  of  the  criteria  used  throughout  the  tables  for  the  analysis  are  the  more  easily 
measurable  indicators  (e.g.  number  of  licenses,  patents,  etc.)  of  commercial  product  development, 
while  there  are  other  relevant  areas  which  are  not  included  in  the  analysis  and  perhaps  hard  to 
measure.  Two  in  particular  are  "impacts"  and  "public  domain  transfers."  Economic  impact  might 
be  under-  or  over-stated  by  these  indicators.  A  product,  software  for  instance,  might  have  lots  of 
licensing  activity  but  a  low  impact  in  its  economic  contribution  to  the  society.  Even  more  important 
is  that  laboratories  participate  in  public  domain  transfers  which  have  had  a  major  economic  impact, 
but  for  which  no  licenses,  fees,  or  other  indicators  in  this  analysis  are  attributable.  An  example 
from  LBL  is  its  development  of  low-emissivity  windows  which  is  in  the  public  domain,  for  which 
we  receive  no  fees  or  licenses,  yet  can  be  attributed  to  a  energy  savings  for  the  nation  of  over  $250 
million  per  year.  A  discussion  somewhere  of  these  limitations  would  make  the  analysis  more 
insightful. 

Along  more  positive  lines,  the  five  categories  of  laboratory  activities  on  page  7  and  8  (basic 
research,  applied  research,  development,  technology  transfer,  and  technical  assistance)  are  a  useful 
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identification  of  different  aspects  of  research  and  development.  We  also  believe  it  is  important  to 
recognize,  as  the  report  does  in  various  places,  that  commercial  product  development  does  not 
insure  that  commercial  application  will  be  achieved.  Even  industrial  R&D  labs  have  a  low  rate  of 
successful  commercial  application. 

The  document  as  a  whole  is  a  very  interesting  presentation  and  use  of  complex  data.  We 
certainly  support  the  need  for  such  analysis  for  the  policy  makers  of  the  U.S.,  and  believe  this  is  a 
worthy  step  in  evaluating  the  relationship  and  role  of  government  and  industry  in  research  and 
development  We  applaud  your  effort  in  addressing  this  difficult  task. 


Sincerely, 


Michael  Chartock 
Acting  Head 


Enclosure 

cc:  C.  Shank 
P.  Oddone 

S.  Samuelson,  Berkeley  Site  Office 
Eileen  Rountree-McLennan,  DOE/OAK 
S.  Buswell,  LM-10 
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The  following  are  gao’s  comments  on  the  June  17,  1994,  letter  from 
Lawrence  Berkeley  Laboratory. 


GAO  Comments 


Although  Lawrence  Berkeley  agreed  with  the  study’s  analytic  framework 
and  with  the  need  for  studies  of  this  type  to  inform  congressional 
policymakers,  the  laboratory  raised  an  issue  about  the  relationship  of  the 
national  laboratories’  role  in  commercial  product  development  to  the 
broader  needs  of  industry  or  the  nation,  which  was  not  addressed  in  the 
report.  This  omission  warrants  clarification. 


General  Comments  The  relationship  of  the  national  laboratories’  role  in  commercial  product 

development  to  the  broader  needs  of  industry  is  an  issue  being  discussed 
in  the  public  debate  about  the  laboratories’  missions  and  structure,  but 
one  that  falls  outside  of  the  study  scope.  The  purpose  of  this  study  was  to 
examine  the  extent  to  which  the  national  laboratories  are  engaged  in  basic 
and  applied  research  or  research  related  to  commercial  product 
development.  Scientific  and  technical  infrastructure,  which  Lawrence 
Berkeley  gives  as  an  example  of  industry  need,  while  important  to  the 
considerations  of  laboratory  mission  and  structure  that  serve  as  the 
study’s  policy  context,  was  not  addressed.  It  was  our  expectation  that  the 
findings  of  this  study  would  serve  as  an  empirical  base  for  designing  a 
study  to  address  this  and  other  institutional  issues. 


Specific  Comments  1.  The  statement  “interest  in  the  current  balance  of  the  research  effort  in 

the  10  laboratories’  research  programs  ....  (and)  the  extent  to  which  the 
national  laboratories  are  engaged  now  in  basic  and  applied  research  or  in 
research  related  to  commercial  product  development. ...”  implies  that 
commercial  product  development  is  an  indirect  consequence  of  laboratory 
R&D-related  activities,  rather  than  a  subset  of  the  broader  research  and 
development  role  of  the  national  laboratories.  That  is  the  meaning  of  the 
phrase  “research  related  to  (emphasis  added)  commercial  product 
development,”  and  the  use  of  the  phrase  “outputs  related  to  (emphasis 
added)  commercial  product  development”  elsewhere  in  the  report.  The 
explicit  definitions  and  discussions  of  cradas  in  the  report  make  it  clear 
that  the  laboratories’  involvement  in  commercial  product  development  is 
limited  to  collaboration  with  industry  partners  in  R&D-related  activities  that 
produce  innovations  with  market  potential  and  that  move  these 
technologies  beyond  the  laboratories’  walls.  These  definitions  assume  that 
the  industry  partner  performs  the  subsequent  research,  testing,  and 
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marketing  activity  that  accomplish  commercial  application.  (See  pp.  6, 12, 
and  15.) 

2.  This  study  was  not  designed  as  a  broad  assessment  of  the  national 
laboratories’  roles,  but  to  examine  the  balance  of  the  laboratories’ 
R&D-related  activities  in  two  meg  or  areas:  basic  and  applied  research  and 
research  related  to  commercial  product  development.  We  looked  at  these 
activities  with  three  types  of  measures,  and  our  conclusions  interpret  our 
findings  for  each  type.  The  conclusion  focuses  on  research  related  to 
commercial  product  development  because  we  found  slightly  more  activity 
in  this  area.  We  amplified  this  conclusion  with  an  interpretation  of  findings 
for  the  other  two  types  of  measures.  A  discussion  of  the  noncommercial 
product  output  of  nuclear  weapons  research  was  not  relevant. 

3.  We  have  added  a  discussion  of  these  limitations  to  the  Methodology 
section.  (See  pp.  7-8.) 
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Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


Now  page  26. 


Lawrence  Livermore  National  lAboratory 

OFFICE  OF  MANAGEMENT  REVIEW 
July  6,  1994 

Mr.  Kwai-Cheung  Chan, 

Director,  Program  Evaluation  in  Physical  Systems  Areas 
U.S.  General  Accounting  Office,  PEMD  Room  5844 
441  G  Street,  NW. 

Washington,  DC  20548 

Re:  GAO  Draft  Report  -  National  Laboratories:  Research  and  Development 

Activities 

Dear  Mr.  Chan: 

Thank  you  for  the  opportunity  to  review  and  comment  on  the  subject  draft  report. 

In  reference  to  Appendix  I,  page  4,  Lawrence  Livermore  National  Laboratory,  we  request 
that  you  consider  including  the  following  revised  description  of  the  Lawrence  Livermore  National 
Laboratory  (revisions  are  shown  in  italics). 

Lawrence  Livermore  was  established  in  1952.  The  University  of  California  operates  the 
laboratory  which  serves  as  a  national  resource  in  science  and  engineering,  focused  on  national 
security,  energy,  the  environment,  biomedicine,  economic  competitiveness,  and  science  and 
mathematics  education,  with  a  special  responsibility  for  nuclear  weapons.  National  security  has 
traditionally  been  a  special  focus  of  the  laboratory's  research  and  development  effort.  Lawrence 
Livermore's  major  areas  of  activity  have  included  research,  development,  and  testing  for  all 
phases  of  the  nuclear  weapons  life  cycle;  strategic  defense  research;  arms  control  and  treaty 
verification  technology;  inertial  confinement  fusion;  other  energy  research;  research  in 
biological,  ecological  atmospheric  and  geophysical  sciences;  charged  particle  beam  and  frce- 
electron  laser  research;  advanced  laser  and  optical  technology  applications;  technology  transfer; 
and  science  education.  The  laboratory  also  participates  in  human  genome  research  as  part  of  a 
nationally  directed  initiative. 

The  above  revision  provides  a  balanced  description  of  the  Laboratory's  current  mission  and  its 
major  activities  during  the  period  encompassed  by  the  study  (fiscal  year  1989-92). 

We  appreciate  your  consideration  of  our  comments  and  those  provided  by  Lawrence  Berkeley  and 
Los  Alamos  National  Laboratories  in  finalizing  your  report  on  research  and  development  activities  at 
the  national  laboratories.  If  you  have  any  questions  or  require  any  additional  information,  please 
contact  Mr.  John  Keane  at  5 10-422-538 1 . 

Sincerely 

Charles  A.  Dobson,  Manager 

Office  of  Management  Review 

cc:  R.  W.  Cochran 

W.  A.  Lokke 
P.  T.  Schafer 
C.  B.  Tarter 

E.  Rountree-McLennan  DOE/OAK 


US/CAD:yw 

An  Equal  Ofxxirltguty  Empkytg  • !  ■  -f  Cal/ww  P( i  tin,  t>i 1 
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The  following  are  gao’s  comments  on  the  July  6, 1994,  letter  from 
Lawrence  Livermore  National  Laboratory. 


GAO  Comments 


We  have  added  the  revised  text  describing  Lawrence  Livermore 
Laboratory  to  appendix  I. 
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Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


Los  Alamos 

NATIONAL  LABORATORY 

Audits  and  Assessments  Office  Data:  June  23,  1994 

Mail  Stop  A1 1 5 

Los  Alamos,  New  Mexico  87545  Peter  to:  DA  A-94- 149 

(5051  665-3104,  FAX  667-7340 


Mr.  Kwai-Cheung  Chan,  Director 
Program  Evaluation  in  Physical  Systems  Areas,  PEMD 
U.  S.  General  Accounting  Office 
441  G  Street  N.W. 

Washington,  D.C.  20548 

Dear  Mr.  Chan: 

GAO  DRAFT  REPORT:  NATIONAL  LABORATORIES  RESEARCH  AND  DEVELOP¬ 
MENT  ACTIVITIES  (973356) 

Thank  you  for  the  opportunity  to  review  and  comment  on  the  subject  draft  report. 

The  only  comments  the  Laboratory  has  are  the  following: 

BACKGROUND 

The  formal  recognition  of  the  legal  division  of  DoD  and  Civilian  control  regarding 
nuclear  weapons  research  and  development  and  manufacture  seem  a  glaring  omission  in 
the  context  of  the  historical  role  of  the  nuclear  weapons  labs  visa  statutory  responsibility. 
Also,  we  would  suggest  that  the  middle  paragraph  on  page  2  be  modified  to  include  the 
words  "nuclear  weapons  research  and  development".  This  will  allow  the  prose  to  be 
consistent  with  the  definitions  of  research  and  development  that  are  defined  elsewhere  in 
the  document. 

If  you  would  like  to  discuss  this  matter  further,  please  feel  free  to  call  me  at  (505)  665-3104. 
Once  again,  let  me  thank  you  for  the  opportunity  to  comment  on  this  draft  audit  report. 


Margaret  R.  Patterson,  Director 
Audits  and  Assessments  Office 

MRP:mcm 


An  Equal  Opportunity  Employer/Operated  by  the  University  of  California 
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The  following  are  gao’s  comments  on  the  June  23, 1994,  letter  from  Los 
Alamos  National  Laboratory. 


GAO  Comments 


We  have  added  a  footnote  to  the  Background  section  discussing  the  legal 
division  of  Department  of  Defense  and  civilian  responsibility  for  nuclear 
weapons  research  and  development.  We  also  expanded  the  phrase  on  page 
2  from  “weapons  development”  to  “nuclear  weapons  research  and 
development.” 
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Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


See  comment  1. 

Now  p.  2. 

See  comment  2. 


Now  p.  8. 

See  comment  3. 

Now  p.  10. 

See  comment  4. 

Now  p.  15. 

See  comment  5. 

Now  p.  28. 

See  comment  6. 


National  Renewable  Energy  Laboratory 

161 7  Cole  Boulevard 
Golden.  Colorado  80401-3393 
1303)  23MOOO 


*N?=L 


June  22,  1994 
Kwai-Cheung  Chan 

Director,  Program  Evaluation  in  Physical  Systems  Areas 
United  States  General  Accounting  Office 
Washington,  D.C.  20548 

Dear  Mr.  Chan: 

After  reviewing  the  draft  report.  National  Laboratories:  Research  and  Development  Activities, 
I  have  the  following  comments. 

Pg.  1:  Refers  "to  baseline  data  for  reports  on  a  number  of  DOE  laboratory  policy  issues."  It 
would  be  nice  to  present  some  of  the  topics  that  these  reports  might  cover  in  the  future. 

Pg.  1,2:  Alludes  that  NREL  is  a  multi-program  laboratory,  which  it  is  not;  it  is  a  "Program- 
directed  Laboratory."  You  might  consider  describing  briefly  some  pre-NREIVSERI  background. 
This  report  tends  to  give  the  impression  that  NREL  sprung  up  out  of  no  where. 

Pg.  11:  A  simple  table,  supplemented  by  some  text,  might  better  convey  the  overall  information 
on  different  types  of  R&D  activities  by  all  of  the  national  labs. 

Pg.  14:  Table  1.  Split  out  the  three  categories  in  the  graph  under  commercial  product 
development:  development,  technical  assistance,  and  technology  transfer  in  order  to  better  portray 
these  categories. 

Pg.  26:  A  table  that  shows  the  percent  of  critical  technologies  for  each  lab  would  be  useful. 

Appendix  1,  Pg.  7:  NREL  was  not  authorized  as  a  DOE  multi-program  laboratory  in  1991. 
President  Bush  designated  it  a  national  laboratory. 

If  you  have  any  questions  or  would  like  to  discuss  this  material  further,  my  phone  number  is  303- 
275-3027. 


Sincerely, 

A  .  I 

S.  Sherman  Feher 
Senior  Planner 


cc.  Duane  Sunderman 


A  Division  of  Midwest  Research  Institute 
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The  following  are  gao’s  comments  on  the  June  22, 1994,  letter  from 
National  Renewable  Energy  Laboratory. 


GAO  Comments 


1.  One  future  report  will  provide  a  descriptive  statistical  analysis  of  the 
technical  and  operating  characteristics  of  the  national  laboratories’  major 
research  facilities.  Other  topics  are  yet  to  be  determined. 

2.  We  have  made  this  correction  to  the  text. 

3.  Graphs  and  tables  are  presented  in  the  section  on  Principal  Findings. 

4.  The  aggregation  in  figure  1  is  intentional.  The  graph  is  designed  to 
illustrate  the  balance  between  the  two  major  areas  of  R&D-related  activity 
we  examined.  The  last  column  of  table  IV.  1,  labeled  “All  Labs,”  presents 
percentages  for  development,  technical  assistance,  and  technology 
transfer  for  the  10  laboratories. 

5.  See  table  VI.3  in  appendix  VI.  Table  VI.4  presents  these  percentages  by 
program  area. 

6.  We  have  made  this  correction  to  the  text. 
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Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


POST  OFFICE  BOX  2009 

OAK  RIDGE  NATIONAL  LABORATORY  oak  fume.  Tennessee  anm 

MANAGED  ST  MART*  MARIETTA  ENERGY  9V9TEM9.  MC. 

FOR  THE  U.  9  DEPARTMENT  OF  WERGY 


June  21, 1994 


Dr.  Kwai-Cheung  Chan 
Director,  Program  Evaluation  in  Physical 
Systems  Areas 
United  States 
General  Accounting  Office 
Washington,  D.C.  20548 

Dear  Dr.  Chan: 

Thank  you  for  the  opportunity  to  comment  on  the  draft  GAO  report  "National 
Laboratories:  Research  And  Development  Activities." 

We  have  been  requested  by  the  Department  of  Energy's,  Oak  Ridge  Operations  Office 
to  send  our  comments  through  them  to  the  Department  for  coordinating  and 
transmission  to  you. 

Sincerely, 


Alvin  W.  Trivelpiece 
Director 


oc:  Rle-RC 
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We  did  not  receive  Oak  Ridge’s  written  comments  from  doe.  We  did 
discuss  Oak  Ridge’s  views  with  laboratory  representatives  by  telephone  on 
June  22  and  July  13  and  19, 1994,  and  we  spoke  with  a  representative  of 
doe’s  Oak  Ridge  Operations  Office  on  July  7, 1994.  We  also  received  new 
output  data  for  Oak  Ridge’s  subprograms  by  facsimile  from 
representatives  of  both  organizations.  A  summary  of  their  comments  and 
our  response  follows. 


GAO  Comments 


Oak  Ridge  raised  two  general  issues.  One  was  the  effect  of  the  study 
sampling  methodology  on  findings  for  the  laboratory’s  outputs  related  to 
commercial  product  development.  Oak  Ridge  took  the  position  that  most 
of  the  laboratory’s  outputs  related  to  commercial  product  development 
were  produced  by  subprograms  not  selected  in  the  study  sample  and, 
consequently,  expressed  the  concern  that  gao’s  findings  for  outputs 
related  to  commercial  product  development  based  on  the  sampled 
subprograms  may  not  be  representative  because  of  this  distribution  of 
outputs  among  all  laboratory  subprograms.  Most  of  these  outputs,  they 
explained,  are  produced  by  programs  that  fall  below  the  $10-million 
threshold  for  inclusion  in  the  survey.  In  fact,  according  to  tabulations  they 
had  performed,  the  sampled  programs,  while  representing  73  percent  of 
the  overall  budget,  produce  only  7  percent  of  the  outputs  in  question. 

Secondly,  Oak  Ridge  thought  that  the  report’s  definitions  and  analyses 
equate  development  work  with  commercial  product  development  and  that 
the  conclusion  based  on  this  definition  is  not  supported  by  the  data  We 
address  these  issues  separately. 

First,  Oak  Ridge  actually  had  identified  two  sources  of  potential 
underreporting:  (1)  data  for  outputs  of  sampled  subprograms  that  were 
not  available  at  the  time  of  the  survey  and  (2)  data  for  outputs  of 
unsampled  programs.  We  agreed  that  additional  data  for  sampled 
subprograms  should  be  added  to  findings  for  Oak  Ridge.  We  requested  and 
received  from  Oak  Ridge  the  new  data  for  the  sampled  subprograms,  and 
we  added  them  to  our  database  and  report  tables. 

We  did  not  add  to  our  database  and  report  tables  the  summary  data  Oak 
Ridge  tabulated  as  total  outputs  (including  unsampled  subprograms)  for 
the  entire  laboratory.  To  have  incorporated  these  data  would  have  violated 
the  sampling  methodology.  Moreover,  without  more  detailed  information 
at  the  subprogram  level,  we  could  not  judge  to  what  extent  these  totals 
represented  outputs  of  research  and  development  programs.  This  was  a 
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matter  of  some  concern  to  us,  particularly  in  light  of  the  large  number  of 
outputs  Oak  Ridge  ascribed  to  the  unsampled  programs. 

Second,  we  disagreed  that  the  report  definitions  and  analyses  equate 
development  work  with  commercial  product  development.  Our 
definitions,  analyses,  and  conclusions  make  it  clear  that  the  laboratories 
were  not  expected  to  produce  commercial  products.  We  defined 
development  as  having  “some  type  of  product  as  the  output  goal 
(emphasis  added),”  but  concluding  “with  a  prototype  rather  than  a  usable 
good.”  Further,  we  point  out  that  “Additional  time,  research,  and  testing 
are  required  to  convert  the  prototype  to  a  weapon  or  commercially  viable 
product.”  The  definitions  of  outputs  related  to  commercial  product 
development,  including  those  for  precompetitive  commercial  products  and 
processes,  state  that  these  outputs  tend  (emphasis  added)  to  arise  from 
development  work,  but  that  “they  will  require  a  substantial  additional 
investment  before  they  are  ready  to  market.”  The  conclusion,  moreover, 
reiterates  that  these  outputs  are  “precursors  to  marketable  goods,”  and 
that,  for  this  reason,  “it  is  too  early  to  determine  whether  this  activity  will 
produce  technologies  with  commercial  uses.” 

We  also  examined  the  assumption  that  r&d  is  a  linear  process,  with  all 
commercial  product-related  outputs  arising  from  development,  and  found 
that  our  data  did  not  support  it.  We  included  this  segment  of  the  analysis 
to  emphasize  the  uncertainty  associated  with  current  understanding  of  the 
operation  of  the  r&d  process,  and  the  origin  of  technologies  with 
commercial  potential.  The  conclusion  we  reached  concerning  the 
uncertain  prospects  of  the  laboratories’  commercial  product-related 
outputs  is  an  interpretation  of  this  finding  as  well  as  our  definitions  for 
outputs  related  to  commercial  product  development. 
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A  representative  of  Pacific  Northwest,  called  us  on  June  22, 1994,  to 
comment  on  the  draft  report  by  telephone.  A  summary  of  the  laboratory’s 
comments  is  included  in  our  response,  which  follows. 

GAO  Comments 

Pacific  Northwest  offered  one  general  comment  and  several  comments 
and  questions  about  specific  items  in  the  text.  We  address  the  general 
comment  first  and  then  the  specific  comments. 

General  Comment 

Pacific  Northwest  suggested  that  a  section  be  added  to  the  report 
describing  the  major  commercial  product-related  initiatives  the  national 
laboratories  have  undertaken  since  the  end  of  fiscal  year  1992.  Partnership 
for  a  New  Generation  of  Vehicles  (pngv)  and  American  Textile  Partnership 
(amtex),  two  consortia  for  r&d  targeted  on  commercial  applications  in 
which  several  laboratories  are  participating,  were  mentioned  as  examples. 

We  are  aware  that  the  laboratories  have  been  active  in  technology  transfer 
activities  of  many  types  since  the  end  of  fiscal  year  1992.  This  activity  will 
be  captured  in  any  follow-up  study  that  is  performed  in  the  next  few  years 
to  determine  if  progress  has  been  made  since  fiscal  years  1989-92,  the  time 
period  measured  in  this  report. 

Specific  Comments 

1.  Pacific  Northwest  thought  that  the  word  “primarily”  in  the  sentence 
beginning  on  draft  line  10,  page  4  (now  line  12,  p.  3),  should  be  deleted 
because  it  implies  that  the  laboratories  have  only  one  primary  mission.  We 
have  modified  this  sentence. 

2.  Pacific  Northwest  said  that  the  output  data  in  table  5  (now  table  V.l)  not 
reported  for  the  laboratory  are  available  and  will  be  submitted  to  us.  We 
received  and  reviewed  the  data,  and  we  added  it  to  table  V.l. 

3.  Pacific  Northwest  said  that  information  on  crada  formation  for  the 
laboratory  as  a  whole  was  submitted  to  us  during  survey  implementation. 
We  confirmed  that  this  information  had  been  received  and  added  it  to 
table  VI.  1. 
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Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


Sandia  National  Laboratories 

P  O  Box  5800  Managed  and  Operated  by  Sandia  Corporation 

Albuquerque.  New  Mexico  87185-0123  t  subsidiary  of  Martin  Marietta  Corporation 


WIMam  W.  HoWt 

Director 
Audit  Center 

July  1. 1994 


Kwai-Cheung  Chan 
Director 

Program  Evaluation  in  Physical  Systems  Areas 
U.S.  General  Accounting  Office 
Washington,  DC  20548 

Dear  Mr.  Chan 

On  behalf  of  Dr.  A1  Narath,  thank  you  for  the  opportunity  to  comment  about  GAO's 
draft  report  on  National  Laboratories:  Research  and  Development  Activities.  The 
report  accurately  depicts  the  contribution  that  DOE's  Weapons  Program  R&D 
activities  have  made  to  the  national  security  mission,  as  well  as  the 
commercialization  process  and  technology  transfer. 

We  believe  the  concept  of  creating  this  document  is  worthwhile,  the  methodologies  for 
collecting  the  data  are  appropriate,  and  we  suggest  further  updating  the  report's 
contents  to  include  the  national  laboratories'  FY  1994  activities.  Since  creation  of  the 
draft  report's  FY  1992  baseline  statistics,  DOE’s  National  Laboratories  have 
significantly  expanded  their  technology  transfer  expertise,  and  reporting  that 
progress  could  further  enhance  the  document's  potential  for  addressing  that  issue. 

During  our  review  of  the  report,  we  particularly  noticed  Table  VL3 — "Distribution  of 
Expenditures  for  Critical  Technologies  Within  Laboratories:  Mean  Percent."  Our 
experience  indicates  that  data  ranges  this  broad  within  the  weapons  laboratories  are 
unusual.  We  suggest  an  additional  review  of  these  data. 

The  attached  addendum  to  this  letter  contains  additional  editing  suggestions  for  the 
report.  If  you  have  any  further  questions,  please  contact  A.  T.  Schwyzer,  (505)  844- 
2403. 


Sincerely, 


Exceptional  Sendee  In  Ihe  National  (merest 
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The  following  are  gao’s  comments  on  the  July  1, 1994,  letter  from  Sandia 
National  Laboratories. 


GAO  Comments 


Sandia  agreed  with  the  report’s  objective,  methodology,  and  conclusion, 
but  made  two  general  comments.  First,  Sandia  suggested  that  the  report 
include  a  description  of  the  national  laboratories’  expanded  efforts  in 
technology  transfer  during  fiscal  years  1993-94.  Second,  Sandia  suggested 
that  we  review  the  substantial  variation  in  the  percentage  of  laboratory 
funds  not  expended  for  critical  technologies  reported  for  Lawrence 
Livermore,  Los  Alamos,  and  Sandia  in  table  VI.3.  Sandia  expected  this 
percentage  to  be  very  similar  for  all  three  laboratories. 

We  are  aware  that  the  national  laboratories  have  been  active  in  technology 
transfer  activities  of  many  types  during  fiscal  years  1993-94,  including 
participation  in  large-scale  r&d  consortia  such  as  pngv  and  amtex.  These 
activities  will  be  captured  in  any  follow-up  study  that  is  performed  during 
the  next  few  years  to  determine  if  progress  has  been  made  since  fiscal 
years  1989-92,  the  time  period  measured  in  this  report. 

We  reviewed  all  responses  by  Lawrence  Livermore,  Los  Alamos,  and 
Sandia  concerning  percent  of  expenditures  for  critical  technologies  and 
funds  not  expended  for  r&d  in  these  areas.  We  found  that  Lawrence 
Livermore  program  managers  allocated  a  percentage  of  funds  expended  to 
the  “other”  categoiy  to  a  much  greater  extent  than  did  program  managers 
at  Sandia  or  Los  Alamos.  We  also  found  considerable  variation  among  all 
laboratories  in  the  proportion  of  expenditures  allocated  to  this  category. 
r&d  activities  specified  in  the  “other”  category  included  items  such  as 
robotics,  special  nuclear  materials,  environmental  r&d,  and  detector 
technology.  Allocations  to  this  category,  and  to  the  energy  technologies 
category,  accounted  for  most  of  the  difference  in  proportion  of  funds  not 
expended  for  critical  technologies  by  Lawrence  Livermore,  Los  Alamos, 
and  Sandia 
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Major  Contributors  to  This  Report 


Program  Evaluation 
and  Methodology 
Division 


Robert  E.  White,  Assistant  Director 
Sara  E.  Edmondson,  Project  Manager 
Dale  W.  Harrison,  Computer  Analyst 
Venkareddy  Chennareddy,  Referencer 
Richard  R.  Scott,  Project  Adviser 
Gerald  L.  Dillingham,  Project  Adviser 
Eric  M.  Larson,  Project  Adviser 
Nancy  A.  Briggs,  Project  Adviser 


Denver  Regional 
Office 


Miguel  A.  Lujan,  Project  Adviser 


(973356) 
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